Part I - Contingency tables
When working with categorical variables, we are usually interested in
the frequencies of the different categories in our sample. To display
data for two or more categorical variables, cross-tabulations, or
contingency tables, are commonly used - with 2 x 2 tables being the
simplest. We can then test whether there is an association between the
row factor and the column factor by a chi-squared test or a
Fisher’s exact test.
To demonstrate the analysis of contingency tables we will use a
dataset provided with the vcd package. You will need to
install this package using the install.packages R
function.
install.packages("vcd")
library(dplyr)
library(ggplot2)
library(rstatix)
library(ggpubr)
The data frame Arthritis should then be accessible which
is described as:-
Data from Koch & Edwards (1988) from a double-blind clinical
trial investigating a new treatment for rheumatoid arthritis.
#let's use the'Arthritis' dataset in the 'vcd' package
library(vcd)
Arthritis
The count function, included in dplyr can
be used to give a tabulation of the values in any column in the data
frame.
##one way table (count of one variable)
count(Arthritis,Sex)
We can quickly visualise these counts as a barplot. A pie
chart is possible, but not generally recommended.
ggplot(Arthritis, aes(x = Sex)) + geom_bar()

The function freq_table is another way of making the
counts and will also add the proportions as an extra column.
#to get proportion of males and females
freq_table(Arthritis,Sex)
NA
The count function can also compare two columns from a
data frame if both columns are given as arguments.
count(Arthritis,Sex,Improved)
These can also be plotted
count(Arthritis,Sex,Improved) %>%
ggplot(aes(x = Sex, fill = Improved, y = n)) + geom_col()

count(Arthritis,Sex,Improved) %>%
ggplot(aes(x = Sex, fill = Improved, y = n)) + geom_col(position = "dodge")

The freq_table function can be used to calculate the
proportions. However, we need to pay attention to the order in which the
variables are specified as this will dictate how the proportions are
calculated
Compare the output of:-
freq_table(Arthritis,Sex, Improved)
to:-
freq_table(Arthritis,Improved, Sex)
Having explored our data, we can now perform statistical testing. The
function chisq_test can be used to assess whether
differences in proportions are significant or not. We actually don’t
need to calculate the proportions; R will do this for us.
However, we need to re-format the data slightly into a wide
table rather than the default long nature of a data frame in
the tidyverse. We the pivot_wider function we
create a two-by-two table that is typically used for a contingency
analysis.
Arthritis %>%
count(Improved, Treatment) %>%
tidyr::pivot_wider(values_from = n,names_from = Improved)
##would also work with names_from Treatment
And now remove the Treatment column as the
chisq_test function is only expecting numeric data.
The results are presented in a tidyverse tibble that and
we can interpret the test statistics and p-value from this table
Arthritis %>%
count(Improved, Treatment) %>%
tidyr::pivot_wider(values_from = n,names_from = Improved) %>%
select(-Treatment) %>%
chisq_test()
Arthritis %>%
count(Improved, Treatment) %>%
tidyr::pivot_wider(values_from = n,names_from = Improved) %>%
select(-Treatment) %>%
chisq_test() %>% expected_freq()
None Some Marked
[1,] 21.5 7.166667 14.33333
[2,] 20.5 6.833333 13.66667
However, the chisq_test function is not appropriate in
all circumstances.
Arthritis %>%
count(Improved, Sex) %>%
tidyr::pivot_wider(values_from = n,names_from = Improved) %>%
select(-Sex) %>%
chisq_test
Warning: Chi-squared approximation may be incorrect
The Fisher test is recommended for tables with low numbers of
observations (e.g. when more than 20% of cells have expected frequencies
< 5)
Arthritis %>%
count(Improved, Sex) %>%
tidyr::pivot_wider(values_from = n,names_from = Improved) %>%
select(-Sex) %>%
fisher_test
NA
Exercise
1- Read the excel file called Ex Biostat P1.xlsx into R
(see below for the required code)
2- Use the counts function to make a cross-tabulation of
Tumour grade against Gender
3- Determine the percentage of Grade III tumors within
females
4- Use the appropriate test to check if the tumor grade depends on
the gender
## the readxl package is required to read xls and xlsx files into R
## However, csv and tsv files are recommended to store data
tab <- readxl::read_xlsx("data/EX Biostat P1.xlsx")
Part II - How to assess normality
We will read some example data to illustrate how one would test for a
normally-distributed variable. This property is important as it
influences which test we should use.
One of the best ways of displaying data is by using a graph. Graphs
can make both simple and complex data easier to understand by making it
easier to spot trends and patterns. We can use plots to view the
distribution of our data (minimum, maximum, mid-point, spread etc) and
to ensure that the values in our dataset seem realistic (e.g. no
outliers). Many statistical tests rely on the assumption that the data
are normally distributed.
The data for this section are to be found in the file
normal_example.csv in the data folder. You
will need to specify the file path accordingly.
library(readr)
df1 <- read_csv("data/normal_example.csv")
We can inspect the data in RStudio and discover that it consists of a
tidy dataset with numeric values in a column called Values
and a column Var to indicate a variable name
(x)
View(df1)
Various graphical methods are available to assess the distribution.
The first of which is a histogram. Here, the data are split
into “bins” (ggplot2 chooses the number of bins) and the
value on the y axis corresponds to the number of
observations in that bin. The user only has to specify the variable to
be plotted, and ggplot2 takes care of the binning. From
this plot we can judge what the average value of the data is, and the
spread.
Why are we using `gghistogram from ggpubr?
Histograms can be made in ggplot2 by using the
geom_hist function. Here we are going to make use of the
gghistogram function from ggpubr.
gghistogram(df1,x="Value")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

A similar option is to produce a density curve with
ggdensity (also from ggpubr). Here the y-axis
is the density of a particular value.
ggdensity(df1, x="Value")

Combining Histograms and Density
When assessing the distribution of a variable, you might be tempted
to plot the histogram and density on the same plot.
gghistogram has the argument add_density,
which should do the job.
gghistogram(df1,x="Value",add_density = TRUE)
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

However, this doesn’t work. If you check the y-axis limits on the
histogram and density plots, you’ll notice they are on a different
scale. Conveniently there is an argument in gghistogram
that solves this issue.
gghistogram(df1,x="Value",
add_density = TRUE,
y="..density..")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

We can add the standard normal curve with the
following:-
gghistogram(df1,x="Value",add_density = TRUE,y="..density..") + stat_overlay_normal_density(col="steelblue",lwd=2,lty=2)
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

A box plot is an excellent way of displaying continuous data when you
are interested in the spread of your data. The box of the box plot
corresponds to the lower and upper quartiles of the respective
observations and the bar within the box, the median. The whiskers of the
box plot correspond to the distance between the lower/upper quartile and
the smaller of: the smaller/largest measurement OR 1.5 times
the inter quartile range. A disadvantage of the box plot is that you
don’t see the exact data points. However, box plots are very useful in
large datasets where plotting all of the data may give an unclear
picture of the shape of your data.
ggboxplot(df1, y = "Value")

A violin plot is sometimes instead of the boxplot to show
density information.
ggviolin(df1, y = "Value")

Individual points can also be added with geom_jitter;
avoiding over-plotting by adding random noise along the x-axis.
ggviolin(df1, y = "Value",add = "jitter")

Finally, we have a “qq-plot” which allows to compare the
quantiles of our dataset against a theoretical normal distribution. If
the majority of points lie on a diagonal line then the data are
approximately normal.
ggqqplot(df1, x="Value")

These graphical methods are by far the easiest way to assess if a
given dataset is normally-distributed.
For “real-life” data, the results are unlikely to give a perfect
plot, so some degree of judgement and prior experience with the data
type are required. Indeed, it should be noted that the dataset
visualised in the above plots was sampled from a normal distribution.
Even then, the plots were not 100% convincing!
Tests for normality
Although their usage is contentious amongst statisticians, there are
a few methods for testing whether variables are normally-distributed or
not. If the p-value is sufficiently small then we conclude that the data
are not normally distributed. However, some statisticians prefer to use
graphical methods and their intuition about the data or prior knowledge
of the data type (e.g. some measures are generally believed to be
normally-distributed)
#shapiro test
#p<0.05 ...difference between data and normality..data not normal
#p>0.05 ...no diff between data and normality ..data normally distributed
df1 %>%
shapiro_test(Value) %>%
mutate(p < 0.05)
NA
Descriptive Statistics
In the accompanying R
course we have seen how to produce summary statistics of columns in
a dataset. For a dataset that is normally-distributed, appropriate
measures of the average and variability are the mean and
standard deviation. Both these functions are available within R
and can be used in conjunction with the summarise function
in dplyr.
df1 %>%
get_summary_stats
df1 %>%
get_summary_stats %>%
select(mean, sd)
Exercise
1- Read the excel file called Ex Biostat P2.xlsx into
R
2- Identify if the age and hospitalization days are normally
distributed
3- Use the appropriate descriptive statistics [mean and SD, or median
and IQR] for each variable
Part III - Significance tests for continuous variables
In this part we will show how to perform tests to compare 1, 2 (or
more) continuous variables. The dataset, provided by MASH at The
University of Sheffield, describes individuals that have been following
different diets and their age and gender. The main goal of interest is
to determine which of three competing diet regimes results in the
greatest weight loss. However, we can use the dataset to demonstrate
other types of test.
diet <- read_csv("data/diet.csv")
diet
One-sample test
The first hypothesis we will test is whether the people in the study
are overweight or not. This first involves some manipulation of the
table to calculate an extra variable; the Body Mass Index (BMI). We will
test if people in our study are overweight, where overweight is defined
as having a BMI over 25.
\(BMI = weight / height^2\)
(where the weight is measured in kg, and the height in
metres)
Exercise
- Add a new variable to the data frame for the BMI of each person
- you might want to do this in multiple steps using the
%>% notation
We can now test our new variable for normality using the plots and
tests from earlier
gghistogram(diet, x = "BMI", ,add_density = TRUE,y="..density..")+ stat_overlay_normal_density(col="steelblue")
The one-sample t-test is implemented in the function
t_test (as are the various types of t-test that we will
see). The variables to be used in the test are defined using R’s formula
~ syntax.
Y ~ X
Where Y is a numeric variable and
X is a categorical variable indicating the
In the case of a one-sample test we are testing one numeric variable
against a known or population mean (mu). This is written
as:-
Y ~ 1
Note that this not the same as testing against a population mean of
1! t_test will provide us with a t test statistic
and a p-value. However, it is up to us to interpret the p-value
according. There is no guarantee that the test we are
conducting is appropriate.
diet %>%
t_test(BMI ~1)
We get a hugely significant result! However, if we look at the
description for t_test it is testing against a population
mean of \(0\). It is no surprise that
we get a significant result! By changing the mu argument we
can perform a test to see if the people in the study are overweight to
begin with (using 25 as the population or known mean).
diet %>%
t_test(BMI ~1,mu = 25,alternative = "greater")
## 25 is the cutoff for overweight
#t.test(diet$BMI, mu=25,alternative = "greater")
Two-sample tests
A two-sample t-test should be used if you want to compare the
measurements of two populations. There are two types of two-sample
t-test: independent (unpaired) and paired (dependent). To make the
correct choice, you need to understand your underlying data.
An independent two-sample t-test is used when the two samples are
independent of each other, e.g. comparing the mean response of two
groups of patients on treatment vs. control in a clinical
trial.
As the name suggests,a paired two-sample t-test is used when the
two samples are paired, e.g. comparing the mean blood pressure of
patients before and after treatment (two measurements per
patient).
Back to our dataset, we might wonder if there is actually any effect
due to diet so we will compare the intial and final weights.
diet %>%
select(contains("weight"))
We will not go through all the plotting options from before, but here
is the histogram and qqplot for the initial weight.
diet %>%
select(contains("weight")) %>%
gghistogram(x = "initial.weight", add_density = TRUE, y = "..density..") + stat_overlay_normal_density(col="red")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

diet %>%
select(contains("weight")) %>%
ggqqplot(x = "initial.weight")

Exercise: Do you think the initial weight variable
is normally-distributed? Vote on wooclap now!
Re-formating the dataset for tidy analysis
The dataset in it’s current form is not suitable for analysis with
tidy methods. Before proceeding we need to re-format the data into two
columns.
- One column containing numeric variables (each value of weight)
- An indicator (or categorical) variable to denote the group each
numeric observation belongs to (initial or final weight)
The pivot_longer function supports this transformation.
As we want to use all columns in our existing dataset we have to use the
everything() shortcut to select the columns. The names of
our new columns can be chosen, but we will chose Y and
X to be consistent with the formula notation introduced
earlier.
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X")
So now Y contains all our observations of weight and
X indicates when the weights were measured. The
gghistogram function we introduced earlier is able to
visualise data in this form, and we can use the facet.by
argument to produce separate plots for each type of weight
measurement.
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
gghistogram(x = "Y", facet.by = "X",add_density = TRUE, y = "..density..") + stat_overlay_normal_density(col="red")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

Similarly, introducing a group_by function will allow us
to create summaries for each group separately.
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
group_by(X) %>%
get_summary_stats()
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
group_by(X) %>%
shapiro_test(Y)
The t_test function requires us to create a
formula
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
t_test(Y ~ X)
## Just to prove that we get the same result as base R
## Will remove before the course
t.test(diet$final.weight, diet$initial.weight)
Welch Two Sample t-test
data: diet$final.weight and diet$initial.weight
t = -3.0342, df = 149.98, p-value = 0.002843
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-6.515747 -1.376358
sample estimates:
mean of x mean of y
68.34342 72.28947
The p-value is significant and shows that overall the weights of
individuals is different before and after diet. However, this
test is not specifically testing for a decrease after the diet
(which we would really hope to be the case). By adding an extra argument
alternative=less we get a different result
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
t_test(Y ~ X,alternative = "less")
t.test(diet$final.weight, diet$initial.weight,alternative = "less")
Welch Two Sample t-test
data: diet$final.weight and diet$initial.weight
t = -3.0342, df = 149.98, p-value = 0.001422
alternative hypothesis: true difference in means is less than 0
95 percent confidence interval:
-Inf -1.793602
sample estimates:
mean of x mean of y
68.34342 72.28947
Easy way of showing stats result on the plot is using
stat_compare_means
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Weight",names_to = "Time") %>%
ggplot(aes(x = Time, y = Weight)) + geom_boxplot() + geom_jitter(width=0.1) + stat_compare_means(method = "t.test",method.args = list(alternative = "less"))

There is also extra information that we could employ; namely that the
measurements of weight are made on the same person
before and after dieting. This is a classic example of when to apply a
paired t-test. Again, we do not need to use a different function to
perform the test; only add an argument paired=TRUE to
t_test.
diet %>%
select(contains("weight")) %>%
tidyr::pivot_longer(everything(),values_to = "Y",names_to = "X") %>%
t_test(Y ~ X,alternative = "less",paired=TRUE)
t.test(diet$final.weight, diet$initial.weight,alternative = "less", paired=TRUE)
Paired t-test
data: diet$final.weight and diet$initial.weight
t = -13.728, df = 75, p-value < 2.2e-16
alternative hypothesis: true mean difference is less than 0
95 percent confidence interval:
-Inf -3.46735
sample estimates:
mean difference
-3.946053
A convenient plot in ggpubr will allow us to visualise
the paired differences
ggpaired(diet, cond1="initial.weight", cond2="final.weight",line.color = "grey") + stat_compare_means(paired=TRUE,method = "t.test")

The Independent t test, with two independent groups
Lets consider that we want to compare whether males or females lost
more weight during the trial. Here we have two groups and these can be
treated as independant variables as different patients belong
to the two groups.
The null hypothesis for such a test would be that the weight
loss is the same between groups male and female. We seek to evidence to
reject this hypothesis by calculating a test statistic. Firstly, we have
to check for normality:-
diet %>%
mutate(weight.loss = initial.weight - final.weight) %>%
ggdensity("weight.loss",color = "gender") + stat_overlay_normal_density()

diet %>%
mutate(weight.loss = initial.weight - final.weight) %>%
group_by(gender) %>%
shapiro_test(weight.loss)
NA
We conclude that the variances are approximately the same and both
variables are normally-distributed
We can use the t_test function to perform an
independant test. As before, the first argument to
t_test is the R formula notation for the test
being performed. However, this time we do not need the
pivot_longer function
This function allows various type of test to be performed by changing
the appropriate arguments (see the help for t.test for
details (?t.test)). For instance, we can tell the test that
we believe our variances are equal or not.
diet %>%
mutate(weight.loss = initial.weight - final.weight) %>%
t_test(weight.loss ~ gender)
We can see that the t-statistic we observe is consistent with the
null hypothesis, that the weight loss in males and females is the same.
That is, the probability of observing a t-statistic of 0.2 or more, or
-0.2 or less, is quite high.
This is not a significant result (p>0.05), so there is no
evidence of a difference in weight loss between males and
females
diet %>%
mutate(weight.loss = initial.weight - final.weight) %>%
ggplot(aes(x = gender, y=weight.loss)) + geom_boxplot() + stat_compare_means(method="t.test")

Non- Parametric alternatives (e.g. the Wilcoxon test)
Being able to use the t_test relies on the your data
being normally-distributed. If we do not sufficient confidence in this
assumption, there are different statistical tests that can be applied.
Rather than calculating and comparing the means and
variances of different groups they are rank-based
methods. However, they still come with a set of assumptions and involve
the generation of test statistics and p-values.
Independent samples = Wilcoxon rank sum test (Mann Whitney U
test)
This test has many different names including the Wilcoxon, Wilcoxon
two sample test, Wilcoxon-Mann-Whitney, Wilcoxon rank sum and the
Mann-Whitney-U test. However, this test should not be confused with the
Wilcoxon signed rank test (which is used for paired tests). To avoid
confusion this test is usually referred to as the Mann-Whitney U test,
which is used when the dependent variable to be examined is continuous
but the assumptions for parametric tests are violated.
The assumptions of the Mann-Whitney U are as follows:
1.The dependent variable is ordinal or continuous.
2.The data consist of a randomly selected sample of independent
observations from two independent groups.
3.The dependent variables for the two independent groups share a
similar shape.
Fortunately, the R developers have made the function to do a
Wilcox-test similar to doing a t-test. The difficulty is in
choosing the correct test to apply - which R will not advise you
on.
Let’s go back to our example of comparing weight loss between groups
male and female. The equivalent non-parametric version of the test we
performed before is:-
diet %>%
mutate(weight.loss = initial.weight - final.weight) %>%
wilcox_test(weight.loss ~ gender)
The wilcox_test is flexible in much the same way that
t_test in. We can switch to applying a paired test by
adding the argument paired=TRUE.
diet %>%
select(initial.weight,final.weight) %>%
tidyr::pivot_longer(everything(),values_to = "Weight",names_to = "Time") %>%
wilcox_test(Weight ~ Time, paired=TRUE)
Compare between more than two groups
Parametric (ANOVA)
The two-sample t-test is useful when we have just two groups of
continuous data to compare. When we want to compare more than two
groups, a one-way ANOVA can be used to simultaneously compare all
groups, rather than carrying out several individual two-sample t-tests.
The main advantage of doing this is that it reduces the number of tests
being carried out, meaning that the type I error rate (the probability
of seeing a significant result just by chance) does not become
inflated.
In order to justify if an ANOVA test is appropriate we have to test
for normality.
diet <- mutate(diet, weight.loss = initial.weight - final.weight)
diet %>%
gghistogram(x = "weight.loss", facet.by = "diet.type", add_density = TRUE, y = "..density..") + stat_overlay_normal_density(col="red")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

group_by(diet, diet.type) %>%
shapiro_test(weight.loss)
diet %>%
ggqqplot(x = "weight.loss", facet.by = "diet.type")

A one-way ANOVA compares group means by partitioning the variation in
the data into between group variance and within group
variance. Like the other statistical tests we have encountered, the
functions in R do the hard work of calculating the statistics. The
anova_test function is a tidy version of the ANOVA
test.
diet %>%
anova_test(weight.loss ~ diet.type)
ANOVA Table (type II tests)
Effect DFn DFd F p p<.05 ges
1 diet.type 2 73 5.383 0.007 * 0.129
When the test provides a significant result (like above) it tells us
that there is at least on difference in the groups. However, it does not
tell us which group is different. For this, we can apply a “post-hoc
test” such as the Tukey test. If anova_test did not produce
a significant p-value, we wouldn’t proceed with this step
diet %>%
tukey_hsd(weight.loss ~ diet.type)
As we have seen previously, a standard method of presenting the
differences between groups is to use the stat_compare_means
function to automatically add p-values to a boxplot or violin plot
ggplot(diet, aes(x = diet.type, y = weight.loss)) + geom_violin() + geom_jitter(width=0.1) + stat_compare_means(method="anova")

However, in the case of more than two groups it will only show a
single p-value from the ANOVA rather than individual comparisons. We can
explicitly list particular contrasts we are interested in.
my_comparisons <- list( c("A", "B"), c("A", "C"), c("B", "C") )
ggplot(diet, aes(x = diet.type, y = weight.loss)) + geom_violin() + geom_jitter() + stat_compare_means(method = "t.test",comparisons = my_comparisons)

Alternatively, we can manually-compute the p-values and add these to
the plot.
stat_res <- diet %>%
tukey_hsd(weight.loss ~ diet.type)
ggplot(diet, aes(x = diet.type, y = weight.loss)) + geom_violin() + stat_pvalue_manual(stat_res, label = "p.adj",y.position = c(11, 13, 15))

Non-Parametric (Kruskal Wallis)
Data that do not meet the assumptions of ANOVA (e.g. normality) can
be tested using a non-parametric alternative. The
Kruskal-Wallis test is derived from the one-way ANOVA, but uses
ranks rather than actual observations. It is also the extension of the
Mann-Whitney U test to greater than two groups.
diet %>%
kruskal_test(weight.loss ~ diet.type)
kruskal.test(weight.loss ~ as.factor(diet.type), data=diet)
Kruskal-Wallis rank sum test
data: weight.loss by as.factor(diet.type)
Kruskal-Wallis chi-squared = 9.4159, df = 2, p-value =
0.009023
Like the one-way ANOVA this will only tell us that at least one group
is different and not specifically which group(s). The post-hoc
dunn.test is recommended which also performs multiple
testing correction.
diet %>%
dunn_test(weight.loss ~ diet.type, p.adjust.method = "bonferroni")
At this point we could be about to recommend diet C to those that
wish to lose weight. However, are there any other factors in the data
that we should be considering? With ggplot2 we can quite
easily visualise the effects of multiple factors on the data. Lets add
both gender and diet type into the plot. It now appears that diet C is
having an effect on males but not females.
ggplot(diet, aes(x = diet.type, y = weight.loss,fill=gender)) + geom_boxplot()

Two-way ANOVA
The formula notation allows us to specify an
interaction between gender and diet type. In other words, we
are looking to see if the effect of diet type is different for males and
females. In R, the formula for an interaction is specified using a
* between the variables that we are interested in assessing
the interaction for.
diet %>%
anova_test(weight.loss ~ diet.type*gender)
ANOVA Table (type II tests)
Effect DFn DFd F p p<.05 ges
1 diet.type 2 70 5.619 0.005 * 0.138000
2 gender 1 70 0.031 0.860 0.000448
3 diet.type:gender 2 70 3.153 0.049 * 0.083000
This tells us that an effect exists between diet type and gender, but
like before we have to run a post-hoc test to discover more
diet %>%
tukey_hsd(weight.loss ~ diet.type*gender)
Repeated Measures
CHECK THIS CODE DOES WHAT WE WANT
We will read a modified version of the diet dataset in order to test
a “repeated measures” analysis. Here we have added a midpoint weight
measurement.
diet2 <- read_csv("data/diet2.csv")
View(diet2)
However, the three measures that we want to compare are given as
columns in the data frame. We cannot express this using the R
~ notation. In other words the data is in wide
format and not long. We can change this using the
tidyr package. This creates a variable for the type of
measurement (initial, mid or
final) and a variable containing the corresponding value.
The default column names are name and value
respectively.
### With "::" you can use a function from a package you haven't loaded yet
diet2 <- diet2 %>% select(id,gender, contains("weight")) %>%
tidyr::pivot_longer(3:5, values_to = "value", names_to = "time_point") %>%
mutate(id = as.factor(id))
anova_test(diet2, dv =value,
wid = id,
within = time_point)
ANOVA Table (type III tests)
$ANOVA
Effect DFn DFd F p p<.05 ges
1 time_point 2 150 150.396 1.44e-36 * 0.04
$`Mauchly's Test for Sphericity`
Effect W p p<.05
1 time_point 0.642 7.56e-08 *
$`Sphericity Corrections`
Effect GGe DF[GG] p[GG] p[GG]<.05 HFe
1 time_point 0.736 1.47, 110.46 1.25e-27 * 0.748
DF[HF] p[HF] p[HF]<.05
1 1.5, 112.13 5.24e-28 *
Exercise
The excel file ‘RCC2’ contains data about the expression levels of
some genes in patients with renal cell carcinoma. In your study, you put
the following hypotheses. Please test those alternative hypotheses and
state whether you will accept or reject each one.
- Females have a higher level of E2F3 than males
- ANXA expression levels vary between unilateral and bilateral
tumors
- Individuals with RCC grade II have different levels of E2F3 than
those with grade III or IV
- The mean value of miR499 decreases significantly after
treatment
- DFFA is higher in patients with grade IV tumors
Solutions
To be revealed during the workshop!
Note that the code is included from the older version of the
course (using base R and ggplot2) as well as rstatix and ggpubr to check
we get the same answers
Exercise 1
###OLD
##1-Read the excel file called 'EX Biostat P1' into R
Biostat1 <- readxl::read_xlsx("data/EX Biostat P1.xlsx")
##2-Make a cross table showing the gender in the rows and tumor grade in the columns
GenderGrade <- table(Biostat1$Gender, Biostat1$`Tumor grade`)
GenderGrade
I II III
Female 29 12 30
Male 18 34 7
##3-Define the percentage of Grade III tumors within females
prop.table(GenderGrade,1)*100 #(42.3%)
I II III
Female 40.84507 16.90141 42.25352
Male 30.50847 57.62712 11.86441
##4-Add a column in the table showing the sum of the 3 grades in males and in females and state the total number of males and females in the sample
addmargins(GenderGrade,2) #(71 Females, 59 males)
I II III Sum
Female 29 12 30 71
Male 18 34 7 59
##5-Use the appropriate test to check if the tumor grade depends on the gender
chisq.test(GenderGrade) #grade depends on gender (significant)
Pearson's Chi-squared test
data: GenderGrade
X-squared = 26.512, df = 2, p-value = 1.75e-06
## NEW
Biostat1 <- readxl::read_xlsx("data/EX Biostat P1.xlsx")
##2-Make a cross table showing the gender in the rows and tumor grade in the columns
count(Biostat1, Gender, `Tumor grade`)
##3-Define the percentage of Grade III tumors within females
freq_table(Biostat1, Gender, `Tumor grade`)
##4-Use the appropriate test to check if the tumor grade depends on the gender
count(Biostat1, Gender, `Tumor grade`) %>%
tidyr::pivot_wider(everything(), names_from = "Gender",values_from = "n") %>%
select(-`Tumor grade`) %>%
chisq_test()
Warning: Specifying the `id_cols` argument by position was deprecated in tidyr 1.3.0.
Please explicitly name `id_cols`, like `id_cols = everything()`.
Exercise 2
## OLD
##1-Read the excel file called 'EX Biostat P2' into R
Biostat2 <- readxl::read_xlsx("data/EX Biostat P2.xlsx")
hist(Biostat2$Age,freq = FALSE)
lines(density(Biostat2$Age))

##2-Identify if the age and hospitalization days are normally distributed
library(ggplot2)
ggplot(Biostat2, aes(x=Age)) + geom_histogram(aes(y=..density..,),binwidth = 2) + geom_density()

ggplot(Biostat2, aes(x="",y=Age)) + geom_violin() + geom_boxplot()

ggplot(Biostat2,aes(sample=Age)) + geom_qq() + geom_qq_line()

shapiro.test(Biostat2$Age)
Shapiro-Wilk normality test
data: Biostat2$Age
W = 0.97405, p-value = 0.6923
# Age normally distributed
ggplot(Biostat2, aes(x=`hospitalization days`)) + geom_histogram(aes(y=..density..)) + geom_density()

ggplot(Biostat2, aes(x="",y=`hospitalization days`)) + geom_violin() + geom_boxplot()

ggplot(Biostat2,aes(sample=`hospitalization days`)) + geom_qq() + geom_qq_line()

shapiro.test(Biostat2$`hospitalization days`)
Shapiro-Wilk normality test
data: Biostat2$`hospitalization days`
W = 0.89026, p-value = 0.006817
# hospitalization days NOT normally distributed
##3-Use the appropriate descriptive statistics [mean+-SD or median (IQ range)] for each variable
library(tidyverse)
summarise(Biostat2, MeanAge = mean(Age), sdAge = sd(Age),
medianHospDays=median(`hospitalization days`),
Q1=quantile(`hospitalization days`, 0.25),
Q3=quantile(`hospitalization days`, 0.75))
#Age: 9.54 +/- 2.89 [mean +- SD] & Hospitalization days: 11.5 (9.00 - 14.25) [median(IQ range)]
###NEW
##1-Read the excel file called 'EX Biostat P2' into R
Biostat2 <- readxl::read_xlsx("data/EX Biostat P2.xlsx")
gghistogram(Biostat2, x = "Age")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

gghistogram(Biostat2, x = "Age", add_density = TRUE, y = "..density..") + stat_overlay_normal_density(col="blue")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

ggqqplot(Biostat2, x = "Age")

Biostat2 %>%
shapiro_test(vars = "Age")
# Age is normally distributed
gghistogram(Biostat2, x = "hospitalization days")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

gghistogram(Biostat2, x = "hospitalization days", add_density = TRUE, y = "..density..") + stat_overlay_normal_density(col="blue")
Warning: Using `bins = 30` by default. Pick better value with the argument `bins`.

ggqqplot(Biostat2, x = "hospitalization days")

Biostat2 %>%
shapiro_test(vars = "hospitalization days")
# hospitalization days NOT normally distributed
get_summary_stats(Biostat2)
NA
LS0tCnRpdGxlOiAiU3RhdGlzdGljYWwgQW5hbHlzaXMgb2YgQmlvbG9naWNhbCBEYXRhIgphdXRob3I6ICJNYXJrIER1bm5pbmcsIE5pYW1oIEVycmluZ3RvbiBhbmQgQXlhIEVsd2F6aXIiCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIkxhc3QgbW9kaWZpZWQ6ICVkICViICVZIilgJwpvdXRwdXQ6IAogIGh0bWxfbm90ZWJvb2s6IAogICAgdG9jOiB5ZXMKICAgIHRvY19mbG9hdDogeWVzCmVkaXRvcl9vcHRpb25zOiAKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCi0tLQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSxtZXNzYWdlPUZBTFNFKQpgYGAKCiMgQmFja2dyb3VuZAoKSW4gdGhpcyBjb3Vyc2Ugd2Ugd2lsbCBleHBsYWluIHRoZSB0aGVvcnkgYmVoaW5kIGNvbW1vbiBzdGF0aXN0aWNhbCB0ZXN0cyB0byBjb21wYXJlIHR3byBvciBtb3JlIGdyb3Vwcy4gV2Ugd2lsbCBpbGx1c3RyYXRlIHRoaXMgdXNpbmcgYmlvbG9naWNhbGx5LW1vdGl2YXRlZCBleGFtcGxlcyBhbmQgUiBjb2RlIHRoYXQgdXNlcyB0aGUgInRpZHl2ZXJzZSIgdGVjaG5pcXVlcyBvZiBkYXRhIG1hbmlwdWxhdGlvbiBhbmQgdmlzdWFsaXphdGlvbi4KClRvIGluc3RhbGwgdGhlIGVudGlyZSBjb2xsZWN0aW9uIG9mIHRpZHl2ZXJzZSBwYWNrYWdlcyAod2hpY2ggaW5jbHVkZXMgb3RoZXIgdXNlZnVsIGRhdGEtcmVsYXRlZCBwYWNrYWdlcyB0aGF0IHdlIHdpbGwgbm90IHVzZSB0b2RheSksIHlvdSBtYXkgd2lzaCB0byB1c2UgdGhlIGZvbGxvd2luZzotCgpgYGB7ciBldmFsPUZBTFNFfQojIyBNYWtlIHRha2Ugc29tZSB0aW1lIQppbnN0YWxsLnBhY2thZ2VzKCJ0aWR5dmVyc2UiKQpgYGAKClN0YXRpc3RpY3MgaGFzIGJlZW4gYSBmdW5kYW1lbnRhbCBwYXJ0IG9mIHRoZSBSIGxhbmd1YWdlIGZyb20gdGhlIGJlZ2lubmluZy4gQSB2YXN0IGFycmF5IG9mIHN0YXRpc3RpY2FsIHRlc3RzIGFuZCBhc3NvY2lhdGVkIHZpc3VhbGl6YXRpb25zIGFyZSBzdXBwb3J0ZWQuIEhvd2V2ZXIsIHRoZSBmdW5jdGlvbnMgdG8gcGVyZm9ybSB0aGVzZSBwcmVkYXRlIHRoZSBjcmVhdGlvbiBvZiB0aGUgYHRpZHl2ZXJzZWAgYW5kIGFyZSB1c2VkIGluIGEgbWFubmVyIHRoYXQgbWlnaHQgYmUgc29tZXdoYXQgY291bnRlci1pbnR1aXRpdmUgdG8gc29tZW9uZSB0aGF0IGlzIGZhbWlsaWFyIHdpdGggdGhlIGB0aWR5dmVyc2VgLiBUaGVyZWZvcmUgd2Ugd2lsbCB1c2UgYSBjb3VwbGUgb2YgYWRkLW9uIHBhY2thZ2VzIHRoYXQgaGF2ZSBiZWVuIGNyZWF0ZWQgdG8gcGVyZm9ybSBzdGF0aXN0aWNzIGluIGEgbWFubmVyIHRoYXQgaXMgY29tcGF0aWJsZSB3aXRoIHRoZSB0aWR5dmVyc2UgZWNvLXN5c3RlbS4KCi0gW3JzdGF0aXg6IFN0YXRpc3RpY2FsIHRlc3RpbmcgZm9yIHRoZSB0aWR5dmVyc2VdKGh0dHBzOi8vcnBrZ3MuZGF0YW5vdmlhLmNvbS9yc3RhdGl4LykKLSBbZ2dwdWJyOiDigJhnZ3Bsb3Qy4oCZIEJhc2VkIFB1YmxpY2F0aW9uIFJlYWR5IFBsb3RzXShodHRwczovL3Jwa2dzLmRhdGFub3ZpYS5jb20vZ2dwdWJyLykKCgpUaGUgc3BlY2lmaWMgc2V0IG9mIHJlcXVpcmVkIHBhY2thZ2VzIGlzIGFzIGZvbGxvd3M6LQoKYGBge3IgZXZhbD1GQUxTRX0KaW5zdGFsbC5wYWNrYWdlcyhjKCJkcGx5ciIsCiAgICAgICAgICAgICAgICAgICAiZ2dwbG90MiIsCiAgICAgICAgICAgICAgICAgICAicmVhZHIiLAogICAgICAgICAgICAgICAgICAgInJlYWR4bCIsCiAgICAgICAgICAgICAgICAgICAicnN0YXRpeCIsCiAgICAgICAgICAgICAgICAgICAiZ2dwdWJyIiwKICAgICAgICAgICAgICAgICAgICJybWFya2Rvd24iLAogICAgICAgICAgICAgICAgICAgInRpZHlyIiwKICAgICAgICAgICAgICAgICAgICJ2Y2QiKSkgCmBgYAoKCgojIFBhcnQgSSAtIENvbnRpbmdlbmN5IHRhYmxlcwoKV2hlbiB3b3JraW5nIHdpdGggY2F0ZWdvcmljYWwgdmFyaWFibGVzLCB3ZSBhcmUgdXN1YWxseSBpbnRlcmVzdGVkIGluIHRoZSBmcmVxdWVuY2llcyBvZiB0aGUgZGlmZmVyZW50IGNhdGVnb3JpZXMgaW4gb3VyIHNhbXBsZS4gVG8gZGlzcGxheSBkYXRhIGZvciB0d28gb3IgbW9yZSBjYXRlZ29yaWNhbCB2YXJpYWJsZXMsIGNyb3NzLXRhYnVsYXRpb25zLCBvciBjb250aW5nZW5jeSB0YWJsZXMsIGFyZSBjb21tb25seSB1c2VkIC0gd2l0aCAyIHggMiB0YWJsZXMgYmVpbmcgdGhlIHNpbXBsZXN0LiBXZSBjYW4gdGhlbiB0ZXN0IHdoZXRoZXIgdGhlcmUgaXMgYW4gYXNzb2NpYXRpb24gYmV0d2VlbiB0aGUgcm93IGZhY3RvciBhbmQgdGhlIGNvbHVtbiBmYWN0b3IgYnkgYSAqY2hpLXNxdWFyZWQgdGVzdCogb3IgYSAqRmlzaGVy4oCZcyBleGFjdCB0ZXN0Ki4KClRvIGRlbW9uc3RyYXRlIHRoZSBhbmFseXNpcyBvZiBjb250aW5nZW5jeSB0YWJsZXMgd2Ugd2lsbCB1c2UgYSBkYXRhc2V0IHByb3ZpZGVkIHdpdGggdGhlIGB2Y2RgIHBhY2thZ2UuIFlvdSB3aWxsIG5lZWQgdG8gaW5zdGFsbCB0aGlzIHBhY2thZ2UgdXNpbmcgdGhlIGBpbnN0YWxsLnBhY2thZ2VzYCBSIGZ1bmN0aW9uLgoKCmBgYHtyIGV2YWw9RkFMU0V9Cmluc3RhbGwucGFja2FnZXMoInZjZCIpCmBgYAoKCmBgYHtyfQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkocnN0YXRpeCkKbGlicmFyeShnZ3B1YnIpCmBgYAoKClRoZSBkYXRhIGZyYW1lIGBBcnRocml0aXNgIHNob3VsZCB0aGVuIGJlIGFjY2Vzc2libGUgd2hpY2ggaXMgZGVzY3JpYmVkIGFzOi0gCgo+IERhdGEgZnJvbSBLb2NoICYgRWR3YXJkcyAoMTk4OCkgZnJvbSBhIGRvdWJsZS1ibGluZCBjbGluaWNhbCB0cmlhbCBpbnZlc3RpZ2F0aW5nIGEgbmV3IHRyZWF0bWVudCBmb3IgcmhldW1hdG9pZCBhcnRocml0aXMuCgpgYGB7cn0KI2xldCdzIHVzZSB0aGUnQXJ0aHJpdGlzJyBkYXRhc2V0IGluIHRoZSAndmNkJyBwYWNrYWdlIApsaWJyYXJ5KHZjZCkKQXJ0aHJpdGlzCmBgYAoKVGhlIGBjb3VudGAgZnVuY3Rpb24sIGluY2x1ZGVkIGluIGBkcGx5cmAgY2FuIGJlIHVzZWQgdG8gZ2l2ZSBhIHRhYnVsYXRpb24gb2YgdGhlIHZhbHVlcyBpbiBhbnkgY29sdW1uIGluIHRoZSBkYXRhIGZyYW1lLgoKYGBge3IgbWVzc2FnZT1GQUxTRX0KIyNvbmUgd2F5IHRhYmxlIChjb3VudCBvZiBvbmUgdmFyaWFibGUpCmNvdW50KEFydGhyaXRpcyxTZXgpCmBgYAoKV2UgY2FuIHF1aWNrbHkgdmlzdWFsaXNlIHRoZXNlIGNvdW50cyBhcyBhICpiYXJwbG90Ki4gQSBwaWUgY2hhcnQgaXMgcG9zc2libGUsIGJ1dCBub3QgZ2VuZXJhbGx5IHJlY29tbWVuZGVkLgoKYGBge3J9CmdncGxvdChBcnRocml0aXMsIGFlcyh4ID0gU2V4KSkgKyBnZW9tX2JhcigpCmBgYAoKVGhlIGZ1bmN0aW9uIGBmcmVxX3RhYmxlYCBpcyBhbm90aGVyIHdheSBvZiBtYWtpbmcgdGhlIGNvdW50cyBhbmQgd2lsbCBhbHNvIGFkZCB0aGUgcHJvcG9ydGlvbnMgYXMgYW4gZXh0cmEgY29sdW1uLgoKYGBge3J9CiN0byBnZXQgcHJvcG9ydGlvbiBvZiBtYWxlcyBhbmQgZmVtYWxlcwpmcmVxX3RhYmxlKEFydGhyaXRpcyxTZXgpIAoKYGBgCgpUaGUgYGNvdW50YCBmdW5jdGlvbiBjYW4gYWxzbyBjb21wYXJlIHR3byBjb2x1bW5zIGZyb20gYSBkYXRhIGZyYW1lIGlmIGJvdGggY29sdW1ucyBhcmUgZ2l2ZW4gYXMgYXJndW1lbnRzLgoKYGBge3J9CmNvdW50KEFydGhyaXRpcyxTZXgsSW1wcm92ZWQpCmBgYAoKVGhlc2UgY2FuIGFsc28gYmUgcGxvdHRlZCAKCmBgYHtyfQpjb3VudChBcnRocml0aXMsU2V4LEltcHJvdmVkKSAlPiUgCmdncGxvdChhZXMoeCA9IFNleCwgZmlsbCA9IEltcHJvdmVkLCB5ID0gbikpICsgZ2VvbV9jb2woKQpgYGAKCgpgYGB7cn0KY291bnQoQXJ0aHJpdGlzLFNleCxJbXByb3ZlZCkgJT4lIApnZ3Bsb3QoYWVzKHggPSBTZXgsIGZpbGwgPSBJbXByb3ZlZCwgeSA9IG4pKSArIGdlb21fY29sKHBvc2l0aW9uID0gImRvZGdlIikKYGBgCgpUaGUgYGZyZXFfdGFibGVgIGZ1bmN0aW9uIGNhbiBiZSB1c2VkIHRvIGNhbGN1bGF0ZSB0aGUgcHJvcG9ydGlvbnMuIEhvd2V2ZXIsIHdlIG5lZWQgdG8gcGF5IGF0dGVudGlvbiB0byB0aGUgb3JkZXIgaW4gd2hpY2ggdGhlIHZhcmlhYmxlcyBhcmUgc3BlY2lmaWVkIGFzIHRoaXMgd2lsbCBkaWN0YXRlIGhvdyB0aGUgcHJvcG9ydGlvbnMgYXJlIGNhbGN1bGF0ZWQKCkNvbXBhcmUgdGhlIG91dHB1dCBvZjotCgpgYGB7cn0KZnJlcV90YWJsZShBcnRocml0aXMsU2V4LCBJbXByb3ZlZCkKYGBgCgp0bzotIAoKYGBge3J9CmZyZXFfdGFibGUoQXJ0aHJpdGlzLEltcHJvdmVkLCBTZXgpCmBgYAoKCkhhdmluZyBleHBsb3JlZCBvdXIgZGF0YSwgd2UgY2FuIG5vdyBwZXJmb3JtIHN0YXRpc3RpY2FsIHRlc3RpbmcuIFRoZSBmdW5jdGlvbiBgY2hpc3FfdGVzdGAgY2FuIGJlIHVzZWQgdG8gYXNzZXNzIHdoZXRoZXIgZGlmZmVyZW5jZXMgaW4gcHJvcG9ydGlvbnMgYXJlIHNpZ25pZmljYW50IG9yIG5vdC4gV2UgYWN0dWFsbHkgZG9uJ3QgbmVlZCB0byBjYWxjdWxhdGUgdGhlIHByb3BvcnRpb25zOyBSIHdpbGwgZG8gdGhpcyBmb3IgdXMuCgpIb3dldmVyLCB3ZSBuZWVkIHRvIHJlLWZvcm1hdCB0aGUgZGF0YSBzbGlnaHRseSBpbnRvIGEgKndpZGUqIHRhYmxlIHJhdGhlciB0aGFuIHRoZSBkZWZhdWx0ICpsb25nKiBuYXR1cmUgb2YgYSBkYXRhIGZyYW1lIGluIHRoZSBgdGlkeXZlcnNlYC4gV2UgdGhlIGBwaXZvdF93aWRlcmAgZnVuY3Rpb24gd2UgY3JlYXRlIGEgdHdvLWJ5LXR3byB0YWJsZSB0aGF0IGlzIHR5cGljYWxseSB1c2VkIGZvciBhIGNvbnRpbmdlbmN5IGFuYWx5c2lzLgoKYGBge3J9CkFydGhyaXRpcyAlPiUgCiAgY291bnQoSW1wcm92ZWQsIFRyZWF0bWVudCkgJT4lIAogIHRpZHlyOjpwaXZvdF93aWRlcih2YWx1ZXNfZnJvbSA9IG4sbmFtZXNfZnJvbSA9IEltcHJvdmVkKQoKIyN3b3VsZCBhbHNvIHdvcmsgd2l0aCBuYW1lc19mcm9tIFRyZWF0bWVudAoKYGBgCgpBbmQgbm93IHJlbW92ZSB0aGUgYFRyZWF0bWVudGAgY29sdW1uIGFzIHRoZSBgY2hpc3FfdGVzdGAgZnVuY3Rpb24gaXMgb25seSBleHBlY3RpbmcgbnVtZXJpYyBkYXRhLgoKVGhlIHJlc3VsdHMgYXJlIHByZXNlbnRlZCBpbiBhIGB0aWR5dmVyc2VgIHRpYmJsZSB0aGF0IGFuZCB3ZSBjYW4gaW50ZXJwcmV0IHRoZSB0ZXN0IHN0YXRpc3RpY3MgYW5kIHAtdmFsdWUgZnJvbSB0aGlzIHRhYmxlCgpgYGB7cn0KQXJ0aHJpdGlzICU+JSAKICBjb3VudChJbXByb3ZlZCwgVHJlYXRtZW50KSAlPiUgCiAgdGlkeXI6OnBpdm90X3dpZGVyKHZhbHVlc19mcm9tID0gbixuYW1lc19mcm9tID0gSW1wcm92ZWQpICU+JSAKc2VsZWN0KC1UcmVhdG1lbnQpICAlPiUgCiAgY2hpc3FfdGVzdCgpCmBgYAoKYGBge3J9CkFydGhyaXRpcyAlPiUgCiAgY291bnQoSW1wcm92ZWQsIFRyZWF0bWVudCkgJT4lIAogIHRpZHlyOjpwaXZvdF93aWRlcih2YWx1ZXNfZnJvbSA9IG4sbmFtZXNfZnJvbSA9IEltcHJvdmVkKSAlPiUgCnNlbGVjdCgtVHJlYXRtZW50KSAgJT4lIAogIGNoaXNxX3Rlc3QoKSAlPiUgZXhwZWN0ZWRfZnJlcSgpCmBgYAoKCkhvd2V2ZXIsIHRoZSBgY2hpc3FfdGVzdGAgZnVuY3Rpb24gaXMgbm90IGFwcHJvcHJpYXRlIGluIGFsbCBjaXJjdW1zdGFuY2VzLgoKYGBge3J9CkFydGhyaXRpcyAlPiUgCiAgY291bnQoSW1wcm92ZWQsIFNleCkgJT4lIAogIHRpZHlyOjpwaXZvdF93aWRlcih2YWx1ZXNfZnJvbSA9IG4sbmFtZXNfZnJvbSA9IEltcHJvdmVkKSAlPiUgCiAgc2VsZWN0KC1TZXgpICAlPiUgCiAgY2hpc3FfdGVzdCAKYGBgCgpUaGUgRmlzaGVyIHRlc3QgaXMgcmVjb21tZW5kZWQgZm9yIHRhYmxlcyB3aXRoIGxvdyBudW1iZXJzIG9mIG9ic2VydmF0aW9ucyAoZS5nLiB3aGVuIG1vcmUgdGhhbiAyMCUgb2YgY2VsbHMgaGF2ZSBleHBlY3RlZCBmcmVxdWVuY2llcyA8IDUpCgpgYGB7cn0KQXJ0aHJpdGlzICU+JSAKICBjb3VudChJbXByb3ZlZCwgU2V4KSAlPiUgCiAgdGlkeXI6OnBpdm90X3dpZGVyKHZhbHVlc19mcm9tID0gbixuYW1lc19mcm9tID0gSW1wcm92ZWQpICU+JSAKIHNlbGVjdCgtU2V4KSAgJT4lIAogIGZpc2hlcl90ZXN0CgpgYGAKCjxkaXYgY2xhc3M9ImFsZXJ0IGFsZXJ0LXdhcm5pbmciPgoKKipFeGVyY2lzZSoqCgoxLSBSZWFkIHRoZSBleGNlbCBmaWxlIGNhbGxlZCBgRXggQmlvc3RhdCBQMS54bHN4YCBpbnRvIFIgKHNlZSBiZWxvdyBmb3IgdGhlIHJlcXVpcmVkIGNvZGUpCgoyLSBVc2UgdGhlIGBjb3VudHNgIGZ1bmN0aW9uIHRvIG1ha2UgYSBjcm9zcy10YWJ1bGF0aW9uIG9mIFR1bW91ciBncmFkZSBhZ2FpbnN0IEdlbmRlcgoKMy0gRGV0ZXJtaW5lIHRoZSAqcGVyY2VudGFnZSogb2YgR3JhZGUgSUlJIHR1bW9ycyB3aXRoaW4gZmVtYWxlcyAKCjQtIFVzZSB0aGUgYXBwcm9wcmlhdGUgdGVzdCB0byBjaGVjayBpZiB0aGUgdHVtb3IgZ3JhZGUgZGVwZW5kcyBvbiB0aGUgZ2VuZGVyIAoKYGBge3IgZXZhbD1GQUxTRX0KIyMgdGhlIHJlYWR4bCBwYWNrYWdlIGlzIHJlcXVpcmVkIHRvIHJlYWQgeGxzIGFuZCB4bHN4IGZpbGVzIGludG8gUgojIyBIb3dldmVyLCBjc3YgYW5kIHRzdiBmaWxlcyBhcmUgcmVjb21tZW5kZWQgdG8gc3RvcmUgZGF0YQoKdGFiIDwtIHJlYWR4bDo6cmVhZF94bHN4KCJkYXRhL0VYIEJpb3N0YXQgUDEueGxzeCIpCgpgYGAKCjwvZGl2PgoKCgoKIyBQYXJ0IElJIC0gSG93IHRvIGFzc2VzcyBub3JtYWxpdHkKCldlIHdpbGwgcmVhZCBzb21lIGV4YW1wbGUgZGF0YSB0byBpbGx1c3RyYXRlIGhvdyBvbmUgd291bGQgdGVzdCBmb3IgYSBub3JtYWxseS1kaXN0cmlidXRlZCB2YXJpYWJsZS4gVGhpcyBwcm9wZXJ0eSBpcyBpbXBvcnRhbnQgYXMgaXQgaW5mbHVlbmNlcyB3aGljaCB0ZXN0IHdlIHNob3VsZCB1c2UuCgpPbmUgb2YgdGhlIGJlc3Qgd2F5cyBvZiBkaXNwbGF5aW5nIGRhdGEgaXMgYnkgdXNpbmcgYSBncmFwaC4gR3JhcGhzIGNhbiBtYWtlIGJvdGggc2ltcGxlIGFuZCBjb21wbGV4IGRhdGEgZWFzaWVyIHRvIHVuZGVyc3RhbmQgYnkgbWFraW5nIGl0IGVhc2llciB0byBzcG90IHRyZW5kcyBhbmQgcGF0dGVybnMuIFdlIGNhbiB1c2UgcGxvdHMgdG8gdmlldyB0aGUgZGlzdHJpYnV0aW9uIG9mIG91ciBkYXRhIChtaW5pbXVtLCBtYXhpbXVtLCBtaWQtcG9pbnQsIHNwcmVhZCBldGMpIGFuZCB0byBlbnN1cmUgdGhhdCB0aGUgdmFsdWVzIGluIG91ciBkYXRhc2V0IHNlZW0gcmVhbGlzdGljIChlLmcuIG5vIG91dGxpZXJzKS4gTWFueSBzdGF0aXN0aWNhbCB0ZXN0cyByZWx5IG9uIHRoZSBhc3N1bXB0aW9uIHRoYXQgdGhlIGRhdGEgYXJlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkLgoKVGhlIGRhdGEgZm9yIHRoaXMgc2VjdGlvbiBhcmUgdG8gYmUgZm91bmQgaW4gdGhlIGZpbGUgYG5vcm1hbF9leGFtcGxlLmNzdmAgaW4gdGhlIGBkYXRhYCBmb2xkZXIuIFlvdSB3aWxsIG5lZWQgdG8gc3BlY2lmeSB0aGUgZmlsZSBwYXRoIGFjY29yZGluZ2x5LgoKYGBge3IgbWVzc2FnZT1GQUxTRX0KbGlicmFyeShyZWFkcikKZGYxIDwtIHJlYWRfY3N2KCJkYXRhL25vcm1hbF9leGFtcGxlLmNzdiIpCmBgYAoKV2UgY2FuIGluc3BlY3QgdGhlIGRhdGEgaW4gUlN0dWRpbyBhbmQgZGlzY292ZXIgdGhhdCBpdCBjb25zaXN0cyBvZiBhIHRpZHkgZGF0YXNldCB3aXRoIG51bWVyaWMgdmFsdWVzIGluIGEgY29sdW1uIGNhbGxlZCBgVmFsdWVzYCBhbmQgYSBjb2x1bW4gYFZhcmAgdG8gaW5kaWNhdGUgYSB2YXJpYWJsZSBuYW1lIChgeGApCgpgYGB7cn0KVmlldyhkZjEpCmBgYAoKVmFyaW91cyBncmFwaGljYWwgbWV0aG9kcyBhcmUgYXZhaWxhYmxlIHRvIGFzc2VzcyB0aGUgZGlzdHJpYnV0aW9uLiBUaGUgZmlyc3Qgb2Ygd2hpY2ggaXMgYSAqaGlzdG9ncmFtKi4gSGVyZSwgdGhlIGRhdGEgYXJlIHNwbGl0IGludG8gImJpbnMiIChgZ2dwbG90MmAgY2hvb3NlcyB0aGUgbnVtYmVyIG9mIGJpbnMpIGFuZCB0aGUgdmFsdWUgb24gdGhlIGB5YCBheGlzIGNvcnJlc3BvbmRzIHRvIHRoZSBudW1iZXIgb2Ygb2JzZXJ2YXRpb25zIGluIHRoYXQgYmluLiBUaGUgdXNlciBvbmx5IGhhcyB0byBzcGVjaWZ5IHRoZSB2YXJpYWJsZSB0byBiZSBwbG90dGVkLCBhbmQgYGdncGxvdDJgIHRha2VzIGNhcmUgb2YgdGhlIGJpbm5pbmcuIEZyb20gdGhpcyBwbG90IHdlIGNhbiBqdWRnZSB3aGF0IHRoZSBhdmVyYWdlIHZhbHVlIG9mIHRoZSBkYXRhIGlzLCBhbmQgdGhlIHNwcmVhZC4KCioqV2h5IGFyZSB3ZSB1c2luZyBgZ2doaXN0b2dyYW0gZnJvbSBnZ3B1YnI/KioKCkhpc3RvZ3JhbXMgY2FuIGJlIG1hZGUgaW4gYGdncGxvdDJgIGJ5IHVzaW5nIHRoZSBgZ2VvbV9oaXN0YCBmdW5jdGlvbi4gSGVyZSB3ZSBhcmUgZ29pbmcgdG8gbWFrZSB1c2Ugb2YgdGhlIGBnZ2hpc3RvZ3JhbWAgZnVuY3Rpb24gZnJvbSBgZ2dwdWJyYC4KCmBgYHtyfQpnZ2hpc3RvZ3JhbShkZjEseD0iVmFsdWUiKQpgYGAKCgpBIHNpbWlsYXIgb3B0aW9uIGlzIHRvIHByb2R1Y2UgYSBkZW5zaXR5IGN1cnZlIHdpdGggYGdnZGVuc2l0eWAgKGFsc28gZnJvbSBgZ2dwdWJyYCkuIEhlcmUgdGhlIHktYXhpcyBpcyB0aGUgKmRlbnNpdHkqIG9mIGEgcGFydGljdWxhciB2YWx1ZS4KCmBgYHtyfQpnZ2RlbnNpdHkoZGYxLCB4PSJWYWx1ZSIpCmBgYAoKIyMjIENvbWJpbmluZyBIaXN0b2dyYW1zIGFuZCBEZW5zaXR5IAoKV2hlbiBhc3Nlc3NpbmcgdGhlIGRpc3RyaWJ1dGlvbiBvZiBhIHZhcmlhYmxlLCB5b3UgbWlnaHQgYmUgdGVtcHRlZCB0byBwbG90IHRoZSBoaXN0b2dyYW0gYW5kIGRlbnNpdHkgb24gdGhlIHNhbWUgcGxvdC4gYGdnaGlzdG9ncmFtYCBoYXMgdGhlIGFyZ3VtZW50IGBhZGRfZGVuc2l0eWAsIHdoaWNoIHNob3VsZCBkbyB0aGUgam9iLgoKYGBge3J9CmdnaGlzdG9ncmFtKGRmMSx4PSJWYWx1ZSIsYWRkX2RlbnNpdHkgPSBUUlVFKQpgYGAKCkhvd2V2ZXIsIHRoaXMgZG9lc24ndCB3b3JrLiBJZiB5b3UgY2hlY2sgdGhlIHktYXhpcyBsaW1pdHMgb24gdGhlIGhpc3RvZ3JhbSBhbmQgZGVuc2l0eSBwbG90cywgeW91J2xsIG5vdGljZSB0aGV5IGFyZSBvbiBhIGRpZmZlcmVudCBzY2FsZS4gQ29udmVuaWVudGx5IHRoZXJlIGlzIGFuIGFyZ3VtZW50IGluIGBnZ2hpc3RvZ3JhbWAgdGhhdCBzb2x2ZXMgdGhpcyBpc3N1ZS4KCgpgYGB7cn0KZ2doaXN0b2dyYW0oZGYxLHg9IlZhbHVlIiwKICAgICAgICAgICAgYWRkX2RlbnNpdHkgPSBUUlVFLAogICAgICAgICAgICB5PSIuLmRlbnNpdHkuLiIpCmBgYAoKV2UgY2FuIGFkZCB0aGUgKnN0YW5kYXJkKiBub3JtYWwgY3VydmUgd2l0aCB0aGUgZm9sbG93aW5nOi0KCmBgYHtyfQpnZ2hpc3RvZ3JhbShkZjEseD0iVmFsdWUiLGFkZF9kZW5zaXR5ID0gVFJVRSx5PSIuLmRlbnNpdHkuLiIpICsgc3RhdF9vdmVybGF5X25vcm1hbF9kZW5zaXR5KGNvbD0ic3RlZWxibHVlIixsd2Q9MixsdHk9MikKYGBgCgoKCgpBIGJveCBwbG90IGlzIGFuIGV4Y2VsbGVudCB3YXkgb2YgZGlzcGxheWluZyBjb250aW51b3VzIGRhdGEgd2hlbiB5b3UgYXJlIGludGVyZXN0ZWQgaW4gdGhlIHNwcmVhZCBvZiB5b3VyIGRhdGEuIFRoZSBib3ggb2YgdGhlIGJveCBwbG90IGNvcnJlc3BvbmRzIHRvIHRoZSBsb3dlciBhbmQgdXBwZXIgcXVhcnRpbGVzIG9mIHRoZSByZXNwZWN0aXZlIG9ic2VydmF0aW9ucyBhbmQgdGhlIGJhciB3aXRoaW4gdGhlIGJveCwgdGhlIG1lZGlhbi4gVGhlIHdoaXNrZXJzIG9mIHRoZSBib3ggcGxvdCBjb3JyZXNwb25kIHRvIHRoZSBkaXN0YW5jZSBiZXR3ZWVuIHRoZSBsb3dlci91cHBlciBxdWFydGlsZSBhbmQgdGhlIHNtYWxsZXIgb2Y6IHRoZSBzbWFsbGVyL2xhcmdlc3QgbWVhc3VyZW1lbnQgKk9SKiAxLjUgdGltZXMgdGhlIGludGVyIHF1YXJ0aWxlIHJhbmdlLiBBIGRpc2FkdmFudGFnZSBvZiB0aGUgYm94IHBsb3QgaXMgdGhhdCB5b3UgZG9u4oCZdCBzZWUgdGhlIGV4YWN0IGRhdGEgcG9pbnRzLiBIb3dldmVyLCBib3ggcGxvdHMgYXJlIHZlcnkgdXNlZnVsIGluIGxhcmdlIGRhdGFzZXRzIHdoZXJlIHBsb3R0aW5nIGFsbCBvZiB0aGUgZGF0YSBtYXkgZ2l2ZSBhbiB1bmNsZWFyIHBpY3R1cmUgb2YgdGhlIHNoYXBlIG9mIHlvdXIgZGF0YS4KCmBgYHtyfQoKZ2dib3hwbG90KGRmMSwgeSA9ICJWYWx1ZSIpCmBgYAoKQSAqdmlvbGluIHBsb3QqIGlzIHNvbWV0aW1lcyBpbnN0ZWFkIG9mIHRoZSBib3hwbG90IHRvIHNob3cgZGVuc2l0eSBpbmZvcm1hdGlvbi4KCmBgYHtyfQpnZ3Zpb2xpbihkZjEsIHkgPSAiVmFsdWUiKQpgYGAKCkluZGl2aWR1YWwgcG9pbnRzIGNhbiBhbHNvIGJlIGFkZGVkIHdpdGggYGdlb21faml0dGVyYDsgYXZvaWRpbmcgb3Zlci1wbG90dGluZyBieSBhZGRpbmcgcmFuZG9tIG5vaXNlIGFsb25nIHRoZSB4LWF4aXMuCgpgYGB7cn0KZ2d2aW9saW4oZGYxLCB5ID0gIlZhbHVlIixhZGQgPSAiaml0dGVyIikKYGBgCgoKRmluYWxseSwgd2UgaGF2ZSBhICIqcXEtcGxvdCoiIHdoaWNoIGFsbG93cyB0byBjb21wYXJlIHRoZSBxdWFudGlsZXMgb2Ygb3VyIGRhdGFzZXQgYWdhaW5zdCBhIHRoZW9yZXRpY2FsIG5vcm1hbCBkaXN0cmlidXRpb24uIElmIHRoZSBtYWpvcml0eSBvZiBwb2ludHMgbGllIG9uIGEgZGlhZ29uYWwgbGluZSB0aGVuIHRoZSBkYXRhIGFyZSBhcHByb3hpbWF0ZWx5IG5vcm1hbC4KCmBgYHtyfQpnZ3FxcGxvdChkZjEsIHg9IlZhbHVlIikKYGBgCgpUaGVzZSBncmFwaGljYWwgbWV0aG9kcyBhcmUgYnkgZmFyIHRoZSBlYXNpZXN0IHdheSB0byBhc3Nlc3MgaWYgYSBnaXZlbiBkYXRhc2V0IGlzIG5vcm1hbGx5LWRpc3RyaWJ1dGVkLgoKRm9yICJyZWFsLWxpZmUiIGRhdGEsIHRoZSByZXN1bHRzIGFyZSB1bmxpa2VseSB0byBnaXZlIGEgcGVyZmVjdCBwbG90LCBzbyBzb21lIGRlZ3JlZSBvZiBqdWRnZW1lbnQgYW5kIHByaW9yIGV4cGVyaWVuY2Ugd2l0aCB0aGUgZGF0YSB0eXBlIGFyZSByZXF1aXJlZC4gIEluZGVlZCwgaXQgc2hvdWxkIGJlIG5vdGVkIHRoYXQgdGhlIGRhdGFzZXQgdmlzdWFsaXNlZCBpbiB0aGUgYWJvdmUgcGxvdHMgd2FzIHNhbXBsZWQgZnJvbSBhIG5vcm1hbCBkaXN0cmlidXRpb24uIEV2ZW4gdGhlbiwgdGhlIHBsb3RzIHdlcmUgbm90IDEwMCUgY29udmluY2luZyEKCiMjIFRlc3RzIGZvciBub3JtYWxpdHkKCkFsdGhvdWdoIHRoZWlyIHVzYWdlIGlzIGNvbnRlbnRpb3VzIGFtb25nc3Qgc3RhdGlzdGljaWFucywgdGhlcmUgYXJlIGEgZmV3IG1ldGhvZHMgZm9yIHRlc3Rpbmcgd2hldGhlciB2YXJpYWJsZXMgYXJlIG5vcm1hbGx5LWRpc3RyaWJ1dGVkIG9yIG5vdC4gSWYgdGhlIHAtdmFsdWUgaXMgc3VmZmljaWVudGx5IHNtYWxsIHRoZW4gd2UgY29uY2x1ZGUgdGhhdCB0aGUgZGF0YSBhcmUgbm90IG5vcm1hbGx5IGRpc3RyaWJ1dGVkLiBIb3dldmVyLCBzb21lIHN0YXRpc3RpY2lhbnMgcHJlZmVyIHRvIHVzZSBncmFwaGljYWwgbWV0aG9kcyBhbmQgdGhlaXIgaW50dWl0aW9uIGFib3V0IHRoZSBkYXRhIG9yIHByaW9yIGtub3dsZWRnZSBvZiB0aGUgZGF0YSB0eXBlIChlLmcuIHNvbWUgbWVhc3VyZXMgYXJlIGdlbmVyYWxseSBiZWxpZXZlZCB0byBiZSBub3JtYWxseS1kaXN0cmlidXRlZCkKCmBgYHtyfQoKI3NoYXBpcm8gdGVzdAojcDwwLjA1IC4uLmRpZmZlcmVuY2UgYmV0d2VlbiBkYXRhIGFuZCBub3JtYWxpdHkuLmRhdGEgbm90IG5vcm1hbAojcD4wLjA1IC4uLm5vIGRpZmYgYmV0d2VlbiBkYXRhIGFuZCBub3JtYWxpdHkgLi5kYXRhIG5vcm1hbGx5IGRpc3RyaWJ1dGVkCgpkZjEgJT4lIApzaGFwaXJvX3Rlc3QoVmFsdWUpICU+JSAKICBtdXRhdGUocCA8IDAuMDUpCgpgYGAKCiMjIERlc2NyaXB0aXZlIFN0YXRpc3RpY3MKCkluIHRoZSBbYWNjb21wYW55aW5nIFIgY291cnNlXShodHRwOi8vc2JjLnNoZWYuYWMudWsvci1jcmFzaC1jb3Vyc2UvKSB3ZSBoYXZlIHNlZW4gaG93IHRvIHByb2R1Y2Ugc3VtbWFyeSBzdGF0aXN0aWNzIG9mIGNvbHVtbnMgaW4gYSBkYXRhc2V0LiBGb3IgYSBkYXRhc2V0IHRoYXQgaXMgbm9ybWFsbHktZGlzdHJpYnV0ZWQsIGFwcHJvcHJpYXRlIG1lYXN1cmVzIG9mIHRoZSBhdmVyYWdlIGFuZCB2YXJpYWJpbGl0eSBhcmUgdGhlICptZWFuKiBhbmQgKnN0YW5kYXJkIGRldmlhdGlvbiouIEJvdGggdGhlc2UgZnVuY3Rpb25zIGFyZSBhdmFpbGFibGUgd2l0aGluIFIgYW5kIGNhbiBiZSB1c2VkIGluIGNvbmp1bmN0aW9uIHdpdGggdGhlIGBzdW1tYXJpc2VgIGZ1bmN0aW9uIGluIGBkcGx5cmAuCgpgYGB7cn0KZGYxICU+JSAKICBnZXRfc3VtbWFyeV9zdGF0cyAKYGBgCgpgYGB7cn0KZGYxICU+JSAKICBnZXRfc3VtbWFyeV9zdGF0cyAlPiUgCiAgc2VsZWN0KG1lYW4sIHNkKQpgYGAKCgo8ZGl2IGNsYXNzPSJhbGVydCBhbGVydC13YXJuaW5nIj4KKipFeGVyY2lzZSoqCgoxLSBSZWFkIHRoZSBleGNlbCBmaWxlIGNhbGxlZCBgRXggQmlvc3RhdCBQMi54bHN4YCBpbnRvIFIgCgoyLSBJZGVudGlmeSBpZiB0aGUgYWdlIGFuZCBob3NwaXRhbGl6YXRpb24gZGF5cyBhcmUgbm9ybWFsbHkgZGlzdHJpYnV0ZWQKCjMtIFVzZSB0aGUgYXBwcm9wcmlhdGUgZGVzY3JpcHRpdmUgc3RhdGlzdGljcyBbbWVhbiBhbmQgU0QsICBvciBtZWRpYW4gYW5kIElRUl0gZm9yIGVhY2ggdmFyaWFibGUKCjwvZGl2PgoKCgojIFBhcnQgSUlJIC0gU2lnbmlmaWNhbmNlIHRlc3RzIGZvciBjb250aW51b3VzIHZhcmlhYmxlcwoKSW4gdGhpcyBwYXJ0IHdlIHdpbGwgc2hvdyBob3cgdG8gcGVyZm9ybSB0ZXN0cyB0byBjb21wYXJlIDEsIDIgKG9yIG1vcmUpIGNvbnRpbnVvdXMgdmFyaWFibGVzLiBUaGUgZGF0YXNldCwgcHJvdmlkZWQgYnkgTUFTSCBhdCBUaGUgVW5pdmVyc2l0eSBvZiBTaGVmZmllbGQsIGRlc2NyaWJlcyBpbmRpdmlkdWFscyB0aGF0IGhhdmUgYmVlbiBmb2xsb3dpbmcgZGlmZmVyZW50IGRpZXRzIGFuZCB0aGVpciBhZ2UgYW5kIGdlbmRlci4gVGhlIG1haW4gZ29hbCBvZiBpbnRlcmVzdCBpcyB0byBkZXRlcm1pbmUgd2hpY2ggb2YgdGhyZWUgY29tcGV0aW5nIGRpZXQgcmVnaW1lcyByZXN1bHRzIGluIHRoZSBncmVhdGVzdCB3ZWlnaHQgbG9zcy4gSG93ZXZlciwgd2UgY2FuIHVzZSB0aGUgZGF0YXNldCB0byBkZW1vbnN0cmF0ZSBvdGhlciB0eXBlcyBvZiB0ZXN0LgoKYGBge3IgbWVzc2FnZT1GQUxTRX0KZGlldCA8LSByZWFkX2NzdigiZGF0YS9kaWV0LmNzdiIpCmRpZXQKYGBgCgoKCiMjIE9uZS1zYW1wbGUgdGVzdAoKVGhlIGZpcnN0IGh5cG90aGVzaXMgd2Ugd2lsbCB0ZXN0IGlzIHdoZXRoZXIgdGhlIHBlb3BsZSBpbiB0aGUgc3R1ZHkgYXJlIG92ZXJ3ZWlnaHQgb3Igbm90LiBUaGlzIGZpcnN0IGludm9sdmVzIHNvbWUgbWFuaXB1bGF0aW9uIG9mIHRoZSB0YWJsZSB0byBjYWxjdWxhdGUgYW4gZXh0cmEgdmFyaWFibGU7IHRoZSBCb2R5IE1hc3MgSW5kZXggKEJNSSkuIFdlIHdpbGwgdGVzdCBpZiBwZW9wbGUgaW4gb3VyIHN0dWR5IGFyZSBvdmVyd2VpZ2h0LCB3aGVyZSBvdmVyd2VpZ2h0IGlzIGRlZmluZWQgYXMgaGF2aW5nIGEgQk1JIG92ZXIgMjUuCgokQk1JID0gd2VpZ2h0ICAvIGhlaWdodF4yJAoKKCp3aGVyZSB0aGUgd2VpZ2h0IGlzIG1lYXN1cmVkIGluIGtnLCBhbmQgdGhlIGhlaWdodCBpbiBtZXRyZXMqKQoKPGRpdiBjbGFzcz0iYWxlcnQgYWxlcnQtd2FybmluZyI+CioqRXhlcmNpc2UqKgoKLSBBZGQgYSBuZXcgdmFyaWFibGUgdG8gdGhlIGRhdGEgZnJhbWUgZm9yIHRoZSBCTUkgb2YgZWFjaCBwZXJzb24gCiAgICArIHlvdSBtaWdodCB3YW50IHRvIGRvIHRoaXMgaW4gbXVsdGlwbGUgc3RlcHMgdXNpbmcgdGhlIGAlPiVgIG5vdGF0aW9uCjwvZGl2PgoKYGBge3IgZWNobz1GQUxTRX0KZGlldCA8LSBkaWV0ICU+JSAKICBtdXRhdGUoQk1JID1pbml0aWFsLndlaWdodCAvIChoZWlnaHQvMTAwKV4yKQoKYGBgCgpXZSBjYW4gbm93IHRlc3Qgb3VyIG5ldyB2YXJpYWJsZSBmb3Igbm9ybWFsaXR5IHVzaW5nIHRoZSBwbG90cyBhbmQgdGVzdHMgZnJvbSBlYXJsaWVyCgpgYGB7ciBldmFsPUZBTFNFfQpnZ2hpc3RvZ3JhbShkaWV0LCB4ID0gIkJNSSIsICxhZGRfZGVuc2l0eSA9IFRSVUUseT0iLi5kZW5zaXR5Li4iKSsgc3RhdF9vdmVybGF5X25vcm1hbF9kZW5zaXR5KGNvbD0ic3RlZWxibHVlIikKCmBgYAoKVGhlIG9uZS1zYW1wbGUgdC10ZXN0IGlzIGltcGxlbWVudGVkIGluIHRoZSBmdW5jdGlvbiBgdF90ZXN0YCAoYXMgYXJlIHRoZSB2YXJpb3VzIHR5cGVzIG9mIHQtdGVzdCB0aGF0IHdlIHdpbGwgc2VlKS4gVGhlIHZhcmlhYmxlcyB0byBiZSB1c2VkIGluIHRoZSB0ZXN0IGFyZSBkZWZpbmVkIHVzaW5nIFIncyBmb3JtdWxhIGB+YCBzeW50YXguIAoKYFkgfiBYYAoKV2hlcmUgYFlgIGlzIGEgKm51bWVyaWMqIHZhcmlhYmxlIGFuZCBgWGAgaXMgYSBjYXRlZ29yaWNhbCB2YXJpYWJsZSBpbmRpY2F0aW5nIHRoZSAKCkluIHRoZSBjYXNlIG9mIGEgb25lLXNhbXBsZSB0ZXN0IHdlIGFyZSB0ZXN0aW5nIG9uZSBudW1lcmljIHZhcmlhYmxlIGFnYWluc3QgYSBrbm93biBvciBwb3B1bGF0aW9uIG1lYW4gKGBtdWApLiBUaGlzIGlzIHdyaXR0ZW4gYXM6LQoKYFkgfiAxYAoKTm90ZSB0aGF0IHRoaXMgbm90IHRoZSBzYW1lIGFzIHRlc3RpbmcgYWdhaW5zdCBhIHBvcHVsYXRpb24gbWVhbiBvZiAxISBgdF90ZXN0YCB3aWxsIHByb3ZpZGUgdXMgd2l0aCBhICp0KiB0ZXN0IHN0YXRpc3RpYyBhbmQgYSBwLXZhbHVlLiAqKkhvd2V2ZXIsIGl0IGlzIHVwIHRvIHVzIHRvIGludGVycHJldCB0aGUgcC12YWx1ZSBhY2NvcmRpbmcqKi4gVGhlcmUgaXMgbm8gZ3VhcmFudGVlIHRoYXQgdGhlIHRlc3Qgd2UgYXJlIGNvbmR1Y3RpbmcgaXMgYXBwcm9wcmlhdGUuCgpgYGB7cn0KZGlldCAlPiUgCiAgdF90ZXN0KEJNSSB+MSkKYGBgCgpXZSBnZXQgYSBodWdlbHkgc2lnbmlmaWNhbnQgcmVzdWx0ISBIb3dldmVyLCBpZiB3ZSBsb29rIGF0IHRoZSBkZXNjcmlwdGlvbiBmb3IgYHRfdGVzdGAgaXQgaXMgdGVzdGluZyBhZ2FpbnN0IGEgcG9wdWxhdGlvbiBtZWFuIG9mICQwJC4gSXQgaXMgbm8gc3VycHJpc2UgdGhhdCB3ZSBnZXQgYSBzaWduaWZpY2FudCByZXN1bHQhIEJ5IGNoYW5naW5nIHRoZSBgbXVgIGFyZ3VtZW50IHdlIGNhbiBwZXJmb3JtIGEgdGVzdCB0byBzZWUgaWYgdGhlIHBlb3BsZSBpbiB0aGUgc3R1ZHkgYXJlIG92ZXJ3ZWlnaHQgdG8gYmVnaW4gd2l0aCAodXNpbmcgMjUgYXMgdGhlIHBvcHVsYXRpb24gb3Iga25vd24gbWVhbikuCgpgYGB7cn0KZGlldCAlPiUgCiAgdF90ZXN0KEJNSSB+MSxtdSA9IDI1LGFsdGVybmF0aXZlID0gImdyZWF0ZXIiKQojIyAyNSBpcyB0aGUgY3V0b2ZmIGZvciBvdmVyd2VpZ2h0CiN0LnRlc3QoZGlldCRCTUksIG11PTI1LGFsdGVybmF0aXZlID0gImdyZWF0ZXIiKQpgYGAKCgojIyAgVHdvLXNhbXBsZSB0ZXN0cwoKQSB0d28tc2FtcGxlIHQtdGVzdCBzaG91bGQgYmUgdXNlZCBpZiB5b3Ugd2FudCB0byBjb21wYXJlIHRoZSBtZWFzdXJlbWVudHMgb2YgdHdvIHBvcHVsYXRpb25zLiBUaGVyZSBhcmUgdHdvIHR5cGVzIG9mIHR3by1zYW1wbGUgdC10ZXN0OiBpbmRlcGVuZGVudCAodW5wYWlyZWQpIGFuZCBwYWlyZWQgKGRlcGVuZGVudCkuIFRvIG1ha2UgdGhlIGNvcnJlY3QgY2hvaWNlLCB5b3UgbmVlZCB0byB1bmRlcnN0YW5kIHlvdXIgdW5kZXJseWluZyBkYXRhLiAKCi0gQW4gaW5kZXBlbmRlbnQgdHdvLXNhbXBsZSB0LXRlc3QgaXMgdXNlZCB3aGVuIHRoZSB0d28gc2FtcGxlcyBhcmUgaW5kZXBlbmRlbnQgb2YgZWFjaCBvdGhlciwgZS5nLiAqY29tcGFyaW5nIHRoZSBtZWFuIHJlc3BvbnNlIG9mIHR3byBncm91cHMgb2YgcGF0aWVudHMgb24gdHJlYXRtZW50IHZzLiBjb250cm9sIGluIGEgY2xpbmljYWwgdHJpYWwqLiAKCi0gQXMgdGhlIG5hbWUgc3VnZ2VzdHMsYSBwYWlyZWQgdHdvLXNhbXBsZSB0LXRlc3QgaXMgdXNlZCB3aGVuIHRoZSB0d28gc2FtcGxlcyBhcmUgcGFpcmVkLCBlLmcuICpjb21wYXJpbmcgdGhlIG1lYW4gYmxvb2QgcHJlc3N1cmUgb2YgcGF0aWVudHMgYmVmb3JlIGFuZCBhZnRlciB0cmVhdG1lbnQqICh0d28gbWVhc3VyZW1lbnRzIHBlciBwYXRpZW50KS4KCkJhY2sgdG8gb3VyIGRhdGFzZXQsIHdlIG1pZ2h0IHdvbmRlciBpZiB0aGVyZSBpcyBhY3R1YWxseSBhbnkgZWZmZWN0IGR1ZSB0byBkaWV0IHNvIHdlIHdpbGwgY29tcGFyZSB0aGUgaW50aWFsIGFuZCBmaW5hbCB3ZWlnaHRzLgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpIApgYGAKV2Ugd2lsbCBub3QgZ28gdGhyb3VnaCBhbGwgdGhlIHBsb3R0aW5nIG9wdGlvbnMgZnJvbSBiZWZvcmUsIGJ1dCBoZXJlIGlzIHRoZSBoaXN0b2dyYW0gYW5kIHFxcGxvdCBmb3IgdGhlIGluaXRpYWwgd2VpZ2h0LgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpICU+JSAKICBnZ2hpc3RvZ3JhbSh4ID0gImluaXRpYWwud2VpZ2h0IiwgYWRkX2RlbnNpdHkgPSBUUlVFLCB5ID0gIi4uZGVuc2l0eS4uIikgKyBzdGF0X292ZXJsYXlfbm9ybWFsX2RlbnNpdHkoY29sPSJyZWQiKQpgYGAKCmBgYHtyfQpkaWV0ICU+JSAKICBzZWxlY3QoY29udGFpbnMoIndlaWdodCIpKSAlPiUgCiAgZ2dxcXBsb3QoeCA9ICJpbml0aWFsLndlaWdodCIpCmBgYAo8ZGl2IGNsYXNzPSJleGVyY2lzZSI+CioqRXhlcmNpc2U6KiogRG8geW91IHRoaW5rIHRoZSBpbml0aWFsIHdlaWdodCB2YXJpYWJsZSBpcyBub3JtYWxseS1kaXN0cmlidXRlZD8gVm90ZSBvbiB3b29jbGFwIG5vdyEKPC9kaXY+CgoKIyMgUmUtZm9ybWF0aW5nIHRoZSBkYXRhc2V0IGZvciB0aWR5IGFuYWx5c2lzCgpUaGUgZGF0YXNldCBpbiBpdCdzIGN1cnJlbnQgZm9ybSBpcyBub3Qgc3VpdGFibGUgZm9yIGFuYWx5c2lzIHdpdGggdGlkeSBtZXRob2RzLiBCZWZvcmUgcHJvY2VlZGluZyB3ZSBuZWVkIHRvIHJlLWZvcm1hdCB0aGUgZGF0YSBpbnRvIHR3byBjb2x1bW5zLiAKCi0gT25lIGNvbHVtbiBjb250YWluaW5nIG51bWVyaWMgdmFyaWFibGVzIChlYWNoIHZhbHVlIG9mIHdlaWdodCkKLSBBbiBpbmRpY2F0b3IgKG9yIGNhdGVnb3JpY2FsKSB2YXJpYWJsZSB0byBkZW5vdGUgdGhlIGdyb3VwIGVhY2ggbnVtZXJpYyBvYnNlcnZhdGlvbiBiZWxvbmdzIHRvIChpbml0aWFsIG9yIGZpbmFsIHdlaWdodCkKClRoZSBgcGl2b3RfbG9uZ2VyYCBmdW5jdGlvbiBzdXBwb3J0cyB0aGlzIHRyYW5zZm9ybWF0aW9uLiBBcyB3ZSB3YW50IHRvIHVzZSBhbGwgY29sdW1ucyBpbiBvdXIgZXhpc3RpbmcgZGF0YXNldCB3ZSBoYXZlIHRvIHVzZSB0aGUgYGV2ZXJ5dGhpbmcoKWAgc2hvcnRjdXQgdG8gc2VsZWN0IHRoZSBjb2x1bW5zLiBUaGUgbmFtZXMgb2Ygb3VyIG5ldyBjb2x1bW5zIGNhbiBiZSBjaG9zZW4sIGJ1dCB3ZSB3aWxsIGNob3NlIGBZYCBhbmQgYFhgIHRvIGJlIGNvbnNpc3RlbnQgd2l0aCB0aGUgZm9ybXVsYSBub3RhdGlvbiBpbnRyb2R1Y2VkIGVhcmxpZXIuCgpgYGB7cn0KZGlldCAlPiUgCiAgc2VsZWN0KGNvbnRhaW5zKCJ3ZWlnaHQiKSkgJT4lIAogIHRpZHlyOjpwaXZvdF9sb25nZXIoZXZlcnl0aGluZygpLHZhbHVlc190byA9ICJZIixuYW1lc190byA9ICJYIikgCmBgYAoKU28gbm93IGBZYCBjb250YWlucyBhbGwgb3VyIG9ic2VydmF0aW9ucyBvZiB3ZWlnaHQgYW5kIGBYYCBpbmRpY2F0ZXMgd2hlbiB0aGUgd2VpZ2h0cyB3ZXJlIG1lYXN1cmVkLiBUaGUgYGdnaGlzdG9ncmFtYCBmdW5jdGlvbiB3ZSBpbnRyb2R1Y2VkIGVhcmxpZXIgaXMgYWJsZSB0byB2aXN1YWxpc2UgZGF0YSBpbiB0aGlzIGZvcm0sIGFuZCB3ZSBjYW4gdXNlIHRoZSBgZmFjZXQuYnlgIGFyZ3VtZW50IHRvIHByb2R1Y2Ugc2VwYXJhdGUgcGxvdHMgZm9yIGVhY2ggdHlwZSBvZiB3ZWlnaHQgbWVhc3VyZW1lbnQuCgpgYGB7cn0KZGlldCAlPiUgCiAgc2VsZWN0KGNvbnRhaW5zKCJ3ZWlnaHQiKSkgJT4lIAogIHRpZHlyOjpwaXZvdF9sb25nZXIoZXZlcnl0aGluZygpLHZhbHVlc190byA9ICJZIixuYW1lc190byA9ICJYIikgJT4lIAogIGdnaGlzdG9ncmFtKHggPSAiWSIsIGZhY2V0LmJ5ID0gIlgiLGFkZF9kZW5zaXR5ID0gVFJVRSwgeSA9ICIuLmRlbnNpdHkuLiIpICsgc3RhdF9vdmVybGF5X25vcm1hbF9kZW5zaXR5KGNvbD0icmVkIikKYGBgCgpTaW1pbGFybHksIGludHJvZHVjaW5nIGEgYGdyb3VwX2J5YCBmdW5jdGlvbiB3aWxsIGFsbG93IHVzIHRvIGNyZWF0ZSBzdW1tYXJpZXMgZm9yIGVhY2ggZ3JvdXAgc2VwYXJhdGVseS4KCmBgYHtyfQpkaWV0ICU+JSAKICBzZWxlY3QoY29udGFpbnMoIndlaWdodCIpKSAlPiUgCiAgdGlkeXI6OnBpdm90X2xvbmdlcihldmVyeXRoaW5nKCksdmFsdWVzX3RvID0gIlkiLG5hbWVzX3RvID0gIlgiKSAlPiUgCiAgZ3JvdXBfYnkoWCkgJT4lIAogIGdldF9zdW1tYXJ5X3N0YXRzKCkKYGBgCgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKGV2ZXJ5dGhpbmcoKSx2YWx1ZXNfdG8gPSAiWSIsbmFtZXNfdG8gPSAiWCIpICU+JSAKICBncm91cF9ieShYKSAlPiUgCiAgc2hhcGlyb190ZXN0KFkpCmBgYApUaGUgYHRfdGVzdGAgZnVuY3Rpb24gcmVxdWlyZXMgdXMgdG8gY3JlYXRlIGEgKmZvcm11bGEqCgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKGV2ZXJ5dGhpbmcoKSx2YWx1ZXNfdG8gPSAiWSIsbmFtZXNfdG8gPSAiWCIpICU+JSAKICB0X3Rlc3QoWSB+IFgpCmBgYAoKCmBgYHtyfQojIyBKdXN0IHRvIHByb3ZlIHRoYXQgd2UgZ2V0IHRoZSBzYW1lIHJlc3VsdCBhcyBiYXNlIFIKIyMgV2lsbCByZW1vdmUgYmVmb3JlIHRoZSBjb3Vyc2UKdC50ZXN0KGRpZXQkZmluYWwud2VpZ2h0LCBkaWV0JGluaXRpYWwud2VpZ2h0KQpgYGAKClRoZSBwLXZhbHVlIGlzIHNpZ25pZmljYW50IGFuZCBzaG93cyB0aGF0IG92ZXJhbGwgdGhlIHdlaWdodHMgb2YgaW5kaXZpZHVhbHMgaXMgKmRpZmZlcmVudCogYmVmb3JlIGFuZCBhZnRlciBkaWV0LiBIb3dldmVyLCB0aGlzIHRlc3QgaXMgbm90IHNwZWNpZmljYWxseSB0ZXN0aW5nIGZvciBhICpkZWNyZWFzZSogYWZ0ZXIgdGhlIGRpZXQgKHdoaWNoIHdlIHdvdWxkIHJlYWxseSBob3BlIHRvIGJlIHRoZSBjYXNlKS4gQnkgYWRkaW5nIGFuIGV4dHJhIGFyZ3VtZW50IGBhbHRlcm5hdGl2ZT1sZXNzYCB3ZSBnZXQgYSBkaWZmZXJlbnQgcmVzdWx0CgpgYGB7cn0KZGlldCAlPiUgCiAgc2VsZWN0KGNvbnRhaW5zKCJ3ZWlnaHQiKSkgJT4lIAogIHRpZHlyOjpwaXZvdF9sb25nZXIoZXZlcnl0aGluZygpLHZhbHVlc190byA9ICJZIixuYW1lc190byA9ICJYIikgJT4lIAogIHRfdGVzdChZIH4gWCxhbHRlcm5hdGl2ZSA9ICJsZXNzIikKYGBgCgoKYGBge3J9CnQudGVzdChkaWV0JGZpbmFsLndlaWdodCwgZGlldCRpbml0aWFsLndlaWdodCxhbHRlcm5hdGl2ZSA9ICJsZXNzIikKYGBgCgpFYXN5IHdheSBvZiBzaG93aW5nIHN0YXRzIHJlc3VsdCBvbiB0aGUgcGxvdCBpcyB1c2luZyBgc3RhdF9jb21wYXJlX21lYW5zYAoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKGV2ZXJ5dGhpbmcoKSx2YWx1ZXNfdG8gPSAiV2VpZ2h0IixuYW1lc190byA9ICJUaW1lIikgJT4lIAogIGdncGxvdChhZXMoeCA9IFRpbWUsIHkgPSBXZWlnaHQpKSArIGdlb21fYm94cGxvdCgpICsgZ2VvbV9qaXR0ZXIod2lkdGg9MC4xKSArIHN0YXRfY29tcGFyZV9tZWFucyhtZXRob2QgPSAidC50ZXN0IixtZXRob2QuYXJncyA9IGxpc3QoYWx0ZXJuYXRpdmUgPSAibGVzcyIpKQpgYGAKCgpUaGVyZSBpcyBhbHNvIGV4dHJhIGluZm9ybWF0aW9uIHRoYXQgd2UgY291bGQgZW1wbG95OyBuYW1lbHkgdGhhdCB0aGUgbWVhc3VyZW1lbnRzIG9mIHdlaWdodCBhcmUgbWFkZSAqKm9uIHRoZSBzYW1lIHBlcnNvbioqIGJlZm9yZSBhbmQgYWZ0ZXIgZGlldGluZy4gVGhpcyBpcyBhIGNsYXNzaWMgZXhhbXBsZSBvZiB3aGVuIHRvIGFwcGx5IGEgcGFpcmVkIHQtdGVzdC4gQWdhaW4sIHdlIGRvIG5vdCBuZWVkIHRvIHVzZSBhIGRpZmZlcmVudCBmdW5jdGlvbiB0byBwZXJmb3JtIHRoZSB0ZXN0OyBvbmx5IGFkZCBhbiBhcmd1bWVudCBgcGFpcmVkPVRSVUVgIHRvIGB0X3Rlc3RgLgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChjb250YWlucygid2VpZ2h0IikpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKGV2ZXJ5dGhpbmcoKSx2YWx1ZXNfdG8gPSAiWSIsbmFtZXNfdG8gPSAiWCIpICU+JSAKICB0X3Rlc3QoWSB+IFgsYWx0ZXJuYXRpdmUgPSAibGVzcyIscGFpcmVkPVRSVUUpCmBgYAoKCmBgYHtyfQp0LnRlc3QoZGlldCRmaW5hbC53ZWlnaHQsIGRpZXQkaW5pdGlhbC53ZWlnaHQsYWx0ZXJuYXRpdmUgPSAibGVzcyIsIHBhaXJlZD1UUlVFKQpgYGAKCkEgY29udmVuaWVudCBwbG90IGluIGBnZ3B1YnJgIHdpbGwgYWxsb3cgdXMgdG8gdmlzdWFsaXNlIHRoZSBwYWlyZWQgZGlmZmVyZW5jZXMKCmBgYHtyfQpnZ3BhaXJlZChkaWV0LCBjb25kMT0iaW5pdGlhbC53ZWlnaHQiLCBjb25kMj0iZmluYWwud2VpZ2h0IixsaW5lLmNvbG9yID0gImdyZXkiKSArIHN0YXRfY29tcGFyZV9tZWFucyhwYWlyZWQ9VFJVRSxtZXRob2QgPSAidC50ZXN0IikKYGBgCgoKIyMgVGhlIEluZGVwZW5kZW50IHQgdGVzdCwgd2l0aCB0d28gaW5kZXBlbmRlbnQgZ3JvdXBzIAoKTGV0cyBjb25zaWRlciB0aGF0IHdlIHdhbnQgdG8gY29tcGFyZSB3aGV0aGVyIG1hbGVzIG9yIGZlbWFsZXMgbG9zdCBtb3JlIHdlaWdodCBkdXJpbmcgdGhlIHRyaWFsLiBIZXJlIHdlIGhhdmUgdHdvIGdyb3VwcyBhbmQgdGhlc2UgY2FuIGJlIHRyZWF0ZWQgYXMgKmluZGVwZW5kYW50KiB2YXJpYWJsZXMgYXMgZGlmZmVyZW50IHBhdGllbnRzIGJlbG9uZyB0byB0aGUgdHdvIGdyb3Vwcy4KClRoZSAqbnVsbCBoeXBvdGhlc2lzKiBmb3Igc3VjaCBhIHRlc3Qgd291bGQgYmUgdGhhdCB0aGUgd2VpZ2h0IGxvc3MgaXMgdGhlIHNhbWUgYmV0d2VlbiBncm91cHMgbWFsZSBhbmQgZmVtYWxlLiBXZSBzZWVrIHRvIGV2aWRlbmNlIHRvIHJlamVjdCB0aGlzIGh5cG90aGVzaXMgYnkgY2FsY3VsYXRpbmcgYSB0ZXN0IHN0YXRpc3RpYy4gRmlyc3RseSwgd2UgaGF2ZSB0byBjaGVjayBmb3Igbm9ybWFsaXR5Oi0KCmBgYHtyfQpkaWV0ICU+JSAKbXV0YXRlKHdlaWdodC5sb3NzID0gaW5pdGlhbC53ZWlnaHQgLSBmaW5hbC53ZWlnaHQpICU+JSAKZ2dkZW5zaXR5KCJ3ZWlnaHQubG9zcyIsY29sb3IgPSAiZ2VuZGVyIikgKyBzdGF0X292ZXJsYXlfbm9ybWFsX2RlbnNpdHkoKQoKZGlldCAlPiUgIAogIG11dGF0ZSh3ZWlnaHQubG9zcyA9IGluaXRpYWwud2VpZ2h0IC0gZmluYWwud2VpZ2h0KSAlPiUgCiAgZ3JvdXBfYnkoZ2VuZGVyKSAlPiUgCiAgc2hhcGlyb190ZXN0KHdlaWdodC5sb3NzKQogIApgYGAKCldlIGNvbmNsdWRlIHRoYXQgdGhlIHZhcmlhbmNlcyBhcmUgYXBwcm94aW1hdGVseSB0aGUgc2FtZSBhbmQgYm90aCB2YXJpYWJsZXMgYXJlIG5vcm1hbGx5LWRpc3RyaWJ1dGVkCgoKV2UgY2FuIHVzZSB0aGUgYHRfdGVzdGAgZnVuY3Rpb24gdG8gcGVyZm9ybSBhbiAqaW5kZXBlbmRhbnQqIHRlc3QuIEFzIGJlZm9yZSwgdGhlIGZpcnN0IGFyZ3VtZW50IHRvIGB0X3Rlc3RgIGlzIHRoZSAqUiBmb3JtdWxhKiBub3RhdGlvbiBmb3IgdGhlIHRlc3QgYmVpbmcgcGVyZm9ybWVkLiBIb3dldmVyLCB0aGlzIHRpbWUgd2UgZG8gbm90IG5lZWQgdGhlIGBwaXZvdF9sb25nZXJgIGZ1bmN0aW9uCgpUaGlzIGZ1bmN0aW9uIGFsbG93cyB2YXJpb3VzIHR5cGUgb2YgdGVzdCB0byBiZSBwZXJmb3JtZWQgYnkgY2hhbmdpbmcgdGhlIGFwcHJvcHJpYXRlIGFyZ3VtZW50cyAoc2VlIHRoZSBoZWxwIGZvciBgdC50ZXN0YCBmb3IgZGV0YWlscyAoYD90LnRlc3RgKSkuIEZvciBpbnN0YW5jZSwgd2UgY2FuIHRlbGwgdGhlIHRlc3QgdGhhdCB3ZSBiZWxpZXZlIG91ciB2YXJpYW5jZXMgYXJlIGVxdWFsIG9yIG5vdC4KCgpgYGB7cn0KZGlldCAlPiUgCiAgbXV0YXRlKHdlaWdodC5sb3NzID0gaW5pdGlhbC53ZWlnaHQgLSBmaW5hbC53ZWlnaHQpICU+JSAKICB0X3Rlc3Qod2VpZ2h0Lmxvc3MgfiBnZW5kZXIpCmBgYAoKCldlIGNhbiBzZWUgdGhhdCB0aGUgdC1zdGF0aXN0aWMgd2Ugb2JzZXJ2ZSBpcyBjb25zaXN0ZW50IHdpdGggdGhlIG51bGwgaHlwb3RoZXNpcywgdGhhdCB0aGUgd2VpZ2h0IGxvc3MgaW4gbWFsZXMgYW5kIGZlbWFsZXMgaXMgdGhlIHNhbWUuIFRoYXQgaXMsIHRoZSBwcm9iYWJpbGl0eSBvZiBvYnNlcnZpbmcgYSB0LXN0YXRpc3RpYyBvZiAwLjIgb3IgbW9yZSwgb3IgLTAuMiBvciBsZXNzLCBpcyBxdWl0ZSBoaWdoLgoKKlRoaXMgaXMgbm90IGEgc2lnbmlmaWNhbnQgcmVzdWx0IChwPjAuMDUpLCBzbyB0aGVyZSBpcyBubyBldmlkZW5jZSBvZiBhIGRpZmZlcmVuY2UgaW4gd2VpZ2h0IGxvc3MgYmV0d2VlbiBtYWxlcyBhbmQgZmVtYWxlcyoKCmBgYHtyfQpkaWV0ICU+JSAKICBtdXRhdGUod2VpZ2h0Lmxvc3MgPSBpbml0aWFsLndlaWdodCAtIGZpbmFsLndlaWdodCkgJT4lIAogIGdncGxvdChhZXMoeCA9IGdlbmRlciwgeT13ZWlnaHQubG9zcykpICsgZ2VvbV9ib3hwbG90KCkgKyBzdGF0X2NvbXBhcmVfbWVhbnMobWV0aG9kPSJ0LnRlc3QiKQpgYGAKCiMjIE5vbi0gUGFyYW1ldHJpYyBhbHRlcm5hdGl2ZXMgKGUuZy4gdGhlIFdpbGNveG9uIHRlc3QpCgpCZWluZyBhYmxlIHRvIHVzZSB0aGUgYHRfdGVzdGAgcmVsaWVzIG9uIHRoZSB5b3VyIGRhdGEgYmVpbmcgbm9ybWFsbHktZGlzdHJpYnV0ZWQuIElmIHdlIGRvIG5vdCBzdWZmaWNpZW50IGNvbmZpZGVuY2UgaW4gdGhpcyBhc3N1bXB0aW9uLCB0aGVyZSBhcmUgZGlmZmVyZW50IHN0YXRpc3RpY2FsIHRlc3RzIHRoYXQgY2FuIGJlIGFwcGxpZWQuIFJhdGhlciB0aGFuIGNhbGN1bGF0aW5nIGFuZCBjb21wYXJpbmcgdGhlICptZWFucyogYW5kICp2YXJpYW5jZXMqIG9mIGRpZmZlcmVudCBncm91cHMgdGhleSBhcmUgKnJhbmstYmFzZWQqIG1ldGhvZHMuIEhvd2V2ZXIsIHRoZXkgc3RpbGwgY29tZSB3aXRoIGEgc2V0IG9mIGFzc3VtcHRpb25zIGFuZCBpbnZvbHZlIHRoZSBnZW5lcmF0aW9uIG9mIHRlc3Qgc3RhdGlzdGljcyBhbmQgcC12YWx1ZXMuCgojIyMgSW5kZXBlbmRlbnQgc2FtcGxlcyA9IFdpbGNveG9uIHJhbmsgc3VtIHRlc3QgKE1hbm4gV2hpdG5leSBVIHRlc3QpCgpUaGlzIHRlc3QgaGFzIG1hbnkgZGlmZmVyZW50IG5hbWVzIGluY2x1ZGluZyB0aGUgV2lsY294b24sIFdpbGNveG9uIHR3byBzYW1wbGUgdGVzdCwgV2lsY294b24tTWFubi1XaGl0bmV5LCBXaWxjb3hvbiByYW5rIHN1bSBhbmQgdGhlIE1hbm4tV2hpdG5leS1VIHRlc3QuIEhvd2V2ZXIsIHRoaXMgdGVzdCBzaG91bGQgbm90IGJlIGNvbmZ1c2VkIHdpdGggdGhlIFdpbGNveG9uIHNpZ25lZCByYW5rIHRlc3QgKHdoaWNoIGlzIHVzZWQgZm9yIHBhaXJlZCB0ZXN0cykuIFRvIGF2b2lkIGNvbmZ1c2lvbiB0aGlzIHRlc3QgaXMgdXN1YWxseSByZWZlcnJlZCB0byBhcyB0aGUgTWFubi1XaGl0bmV5IFUgdGVzdCwgd2hpY2ggaXMgdXNlZCB3aGVuIHRoZSBkZXBlbmRlbnQgdmFyaWFibGUgdG8gYmUgZXhhbWluZWQgaXMgY29udGludW91cyBidXQgdGhlIGFzc3VtcHRpb25zIGZvciBwYXJhbWV0cmljIHRlc3RzIGFyZSB2aW9sYXRlZC4KClRoZSBhc3N1bXB0aW9ucyBvZiB0aGUgTWFubi1XaGl0bmV5IFUgYXJlIGFzIGZvbGxvd3M6CgoxLlRoZSBkZXBlbmRlbnQgdmFyaWFibGUgaXMgb3JkaW5hbCBvciBjb250aW51b3VzLgoKMi5UaGUgZGF0YSBjb25zaXN0IG9mIGEgcmFuZG9tbHkgc2VsZWN0ZWQgc2FtcGxlIG9mIGluZGVwZW5kZW50IG9ic2VydmF0aW9ucyBmcm9tIHR3byBpbmRlcGVuZGVudCBncm91cHMuCgozLlRoZSBkZXBlbmRlbnQgdmFyaWFibGVzIGZvciB0aGUgdHdvIGluZGVwZW5kZW50IGdyb3VwcyBzaGFyZSBhIHNpbWlsYXIgc2hhcGUuCgpGb3J0dW5hdGVseSwgdGhlIFIgZGV2ZWxvcGVycyBoYXZlIG1hZGUgdGhlIGZ1bmN0aW9uIHRvIGRvIGEgV2lsY294LXRlc3Qgc2ltaWxhciB0byBkb2luZyBhIHQtdGVzdC4gKipUaGUgZGlmZmljdWx0eSBpcyBpbiBjaG9vc2luZyB0aGUgY29ycmVjdCB0ZXN0IHRvIGFwcGx5IC0gd2hpY2ggUiB3aWxsIG5vdCBhZHZpc2UgeW91IG9uKiouCgpMZXQncyBnbyBiYWNrIHRvIG91ciBleGFtcGxlIG9mIGNvbXBhcmluZyB3ZWlnaHQgbG9zcyBiZXR3ZWVuIGdyb3VwcyBtYWxlIGFuZCBmZW1hbGUuIFRoZSBlcXVpdmFsZW50IG5vbi1wYXJhbWV0cmljIHZlcnNpb24gb2YgdGhlIHRlc3Qgd2UgcGVyZm9ybWVkIGJlZm9yZSBpczotCgpgYGB7cn0KZGlldCAlPiUgCiAgbXV0YXRlKHdlaWdodC5sb3NzID0gaW5pdGlhbC53ZWlnaHQgLSBmaW5hbC53ZWlnaHQpICU+JSAKICB3aWxjb3hfdGVzdCh3ZWlnaHQubG9zcyB+IGdlbmRlcikKYGBgCgpUaGUgYHdpbGNveF90ZXN0YCBpcyBmbGV4aWJsZSBpbiBtdWNoIHRoZSBzYW1lIHdheSB0aGF0IGB0X3Rlc3RgIGluLiBXZSBjYW4gc3dpdGNoIHRvIGFwcGx5aW5nIGEgcGFpcmVkIHRlc3QgYnkgYWRkaW5nIHRoZSBhcmd1bWVudCBgcGFpcmVkPVRSVUVgLgoKYGBge3J9CmRpZXQgJT4lIAogIHNlbGVjdChpbml0aWFsLndlaWdodCxmaW5hbC53ZWlnaHQpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKGV2ZXJ5dGhpbmcoKSx2YWx1ZXNfdG8gPSAiV2VpZ2h0IixuYW1lc190byA9ICJUaW1lIikgJT4lIAogIHdpbGNveF90ZXN0KFdlaWdodCB+IFRpbWUsIHBhaXJlZD1UUlVFKQpgYGAKCgojIyBDb21wYXJlIGJldHdlZW4gKm1vcmUgdGhhbiB0d28qIGdyb3VwcwoKIyMjIyBQYXJhbWV0cmljIChBTk9WQSkKClRoZSB0d28tc2FtcGxlIHQtdGVzdCBpcyB1c2VmdWwgd2hlbiB3ZSBoYXZlIGp1c3QgdHdvIGdyb3VwcyBvZiBjb250aW51b3VzIGRhdGEgdG8gY29tcGFyZS4gV2hlbiB3ZSB3YW50IHRvIGNvbXBhcmUgbW9yZSB0aGFuIHR3byBncm91cHMsIGEgb25lLXdheSBBTk9WQSBjYW4gYmUgdXNlZCB0byBzaW11bHRhbmVvdXNseSBjb21wYXJlIGFsbCBncm91cHMsIHJhdGhlciB0aGFuIGNhcnJ5aW5nIG91dCBzZXZlcmFsIGluZGl2aWR1YWwgdHdvLXNhbXBsZSB0LXRlc3RzLiAgVGhlIG1haW4gYWR2YW50YWdlIG9mIGRvaW5nIHRoaXMgaXMgdGhhdCBpdCByZWR1Y2VzIHRoZSBudW1iZXIgb2YgdGVzdHMgYmVpbmcgY2FycmllZCBvdXQsIG1lYW5pbmcgdGhhdCB0aGUgdHlwZSBJIGVycm9yIHJhdGUgKHRoZSBwcm9iYWJpbGl0eSBvZiBzZWVpbmcgYSBzaWduaWZpY2FudCByZXN1bHQganVzdCBieSBjaGFuY2UpIGRvZXMgbm90IGJlY29tZSBpbmZsYXRlZC4gCgpJbiBvcmRlciB0byBqdXN0aWZ5IGlmIGFuIEFOT1ZBIHRlc3QgaXMgYXBwcm9wcmlhdGUgd2UgaGF2ZSB0byB0ZXN0IGZvciBub3JtYWxpdHkuCgpgYGB7cn0KZGlldCA8LSBtdXRhdGUoZGlldCwgd2VpZ2h0Lmxvc3MgPSBpbml0aWFsLndlaWdodCAtIGZpbmFsLndlaWdodCkKCmRpZXQgJT4lICAgCmdnaGlzdG9ncmFtKHggPSAid2VpZ2h0Lmxvc3MiLCBmYWNldC5ieSA9ICJkaWV0LnR5cGUiLCBhZGRfZGVuc2l0eSA9IFRSVUUsIHkgPSAiLi5kZW5zaXR5Li4iKSArIHN0YXRfb3ZlcmxheV9ub3JtYWxfZGVuc2l0eShjb2w9InJlZCIpCgpncm91cF9ieShkaWV0LCBkaWV0LnR5cGUpICU+JSAKICBzaGFwaXJvX3Rlc3Qod2VpZ2h0Lmxvc3MpCgoKZGlldCAlPiUgCiAgZ2dxcXBsb3QoeCA9ICJ3ZWlnaHQubG9zcyIsIGZhY2V0LmJ5ID0gImRpZXQudHlwZSIpCgpgYGAKCkEgb25lLXdheSBBTk9WQSBjb21wYXJlcyBncm91cCBtZWFucyBieSBwYXJ0aXRpb25pbmcgdGhlIHZhcmlhdGlvbiBpbiB0aGUgZGF0YSBpbnRvICpiZXR3ZWVuIGdyb3VwIHZhcmlhbmNlKiBhbmQgKndpdGhpbiBncm91cCB2YXJpYW5jZSouIExpa2UgdGhlIG90aGVyIHN0YXRpc3RpY2FsIHRlc3RzIHdlIGhhdmUgZW5jb3VudGVyZWQsIHRoZSBmdW5jdGlvbnMgaW4gUiBkbyB0aGUgaGFyZCB3b3JrIG9mIGNhbGN1bGF0aW5nIHRoZSBzdGF0aXN0aWNzLiBUaGUgYGFub3ZhX3Rlc3RgIGZ1bmN0aW9uIGlzIGEgdGlkeSB2ZXJzaW9uIG9mIHRoZSBBTk9WQSB0ZXN0LgoKYGBge3J9CmRpZXQgJT4lIAogIGFub3ZhX3Rlc3Qod2VpZ2h0Lmxvc3MgfiBkaWV0LnR5cGUpCmBgYAoKCldoZW4gdGhlIHRlc3QgcHJvdmlkZXMgYSBzaWduaWZpY2FudCByZXN1bHQgKGxpa2UgYWJvdmUpIGl0IHRlbGxzIHVzIHRoYXQgdGhlcmUgaXMgYXQgbGVhc3Qgb24gZGlmZmVyZW5jZSBpbiB0aGUgZ3JvdXBzLiBIb3dldmVyLCBpdCBkb2VzIG5vdCB0ZWxsIHVzIHdoaWNoIGdyb3VwIGlzIGRpZmZlcmVudC4gRm9yIHRoaXMsIHdlIGNhbiBhcHBseSBhICJwb3N0LWhvYyB0ZXN0IiBzdWNoIGFzIHRoZSBUdWtleSB0ZXN0LiBJZiBgYW5vdmFfdGVzdGAgZGlkIG5vdCBwcm9kdWNlIGEgc2lnbmlmaWNhbnQgcC12YWx1ZSwgd2Ugd291bGRuJ3QgcHJvY2VlZCB3aXRoIHRoaXMgc3RlcAoKYGBge3J9CmRpZXQgJT4lICAKICB0dWtleV9oc2Qod2VpZ2h0Lmxvc3MgfiBkaWV0LnR5cGUpCmBgYAoKQXMgd2UgaGF2ZSBzZWVuIHByZXZpb3VzbHksIGEgc3RhbmRhcmQgbWV0aG9kIG9mIHByZXNlbnRpbmcgdGhlIGRpZmZlcmVuY2VzIGJldHdlZW4gZ3JvdXBzIGlzIHRvIHVzZSB0aGUgYHN0YXRfY29tcGFyZV9tZWFuc2AgZnVuY3Rpb24gdG8gYXV0b21hdGljYWxseSBhZGQgcC12YWx1ZXMgdG8gYSBib3hwbG90IG9yIHZpb2xpbiBwbG90CgpgYGB7cn0KZ2dwbG90KGRpZXQsIGFlcyh4ID0gZGlldC50eXBlLCB5ID0gd2VpZ2h0Lmxvc3MpKSArIGdlb21fdmlvbGluKCkgKyBnZW9tX2ppdHRlcih3aWR0aD0wLjEpICsgc3RhdF9jb21wYXJlX21lYW5zKG1ldGhvZD0iYW5vdmEiKQpgYGAKSG93ZXZlciwgaW4gdGhlIGNhc2Ugb2YgbW9yZSB0aGFuIHR3byBncm91cHMgaXQgd2lsbCBvbmx5IHNob3cgYSBzaW5nbGUgcC12YWx1ZSBmcm9tIHRoZSBBTk9WQSByYXRoZXIgdGhhbiBpbmRpdmlkdWFsIGNvbXBhcmlzb25zLiBXZSBjYW4gZXhwbGljaXRseSBsaXN0IHBhcnRpY3VsYXIgY29udHJhc3RzIHdlIGFyZSBpbnRlcmVzdGVkIGluLgoKYGBge3J9Cm15X2NvbXBhcmlzb25zIDwtIGxpc3QoIGMoIkEiLCAiQiIpLCBjKCJBIiwgIkMiKSwgYygiQiIsICJDIikgKQoKZ2dwbG90KGRpZXQsIGFlcyh4ID0gZGlldC50eXBlLCB5ID0gd2VpZ2h0Lmxvc3MpKSArIGdlb21fdmlvbGluKCkgKyBnZW9tX2ppdHRlcigpICsgIHN0YXRfY29tcGFyZV9tZWFucyhtZXRob2QgPSAidC50ZXN0Iixjb21wYXJpc29ucyA9IG15X2NvbXBhcmlzb25zKQpgYGAKQWx0ZXJuYXRpdmVseSwgd2UgY2FuIG1hbnVhbGx5LWNvbXB1dGUgdGhlIHAtdmFsdWVzIGFuZCBhZGQgdGhlc2UgdG8gdGhlIHBsb3QuCgpgYGB7cn0Kc3RhdF9yZXMgPC0gZGlldCAlPiUgCiAgdHVrZXlfaHNkKHdlaWdodC5sb3NzIH4gZGlldC50eXBlKQoKZ2dwbG90KGRpZXQsIGFlcyh4ID0gZGlldC50eXBlLCB5ID0gd2VpZ2h0Lmxvc3MpKSArIGdlb21fdmlvbGluKCkgKyBzdGF0X3B2YWx1ZV9tYW51YWwoc3RhdF9yZXMsIGxhYmVsID0gInAuYWRqIix5LnBvc2l0aW9uID0gYygxMSwgMTMsIDE1KSkKYGBgCgoKCiMjIyBOb24tUGFyYW1ldHJpYyAoS3J1c2thbCBXYWxsaXMpIAoKRGF0YSB0aGF0IGRvIG5vdCBtZWV0IHRoZSBhc3N1bXB0aW9ucyBvZiBBTk9WQSAoZS5nLiBub3JtYWxpdHkpIGNhbiBiZSB0ZXN0ZWQgdXNpbmcgYSBub24tcGFyYW1ldHJpYyBhbHRlcm5hdGl2ZS4gVGhlICpLcnVza2FsLVdhbGxpcyogdGVzdCBpcyBkZXJpdmVkIGZyb20gdGhlIG9uZS13YXkgQU5PVkEsIGJ1dCB1c2VzIHJhbmtzIHJhdGhlciB0aGFuIGFjdHVhbCBvYnNlcnZhdGlvbnMuIEl0IGlzIGFsc28gdGhlIGV4dGVuc2lvbiBvZiB0aGUgTWFubi1XaGl0bmV5IFUgdGVzdCB0byBncmVhdGVyIHRoYW4gdHdvIGdyb3Vwcy4KCmBgYHtyfQpkaWV0ICU+JSAKICBrcnVza2FsX3Rlc3Qod2VpZ2h0Lmxvc3MgfiBkaWV0LnR5cGUpCmBgYAoKCmBgYHtyfQprcnVza2FsLnRlc3Qod2VpZ2h0Lmxvc3MgfiBhcy5mYWN0b3IoZGlldC50eXBlKSwgZGF0YT1kaWV0KQpgYGAKCkxpa2UgdGhlIG9uZS13YXkgQU5PVkEgdGhpcyB3aWxsIG9ubHkgdGVsbCB1cyB0aGF0IGF0IGxlYXN0IG9uZSBncm91cCBpcyBkaWZmZXJlbnQgYW5kIG5vdCBzcGVjaWZpY2FsbHkgd2hpY2ggZ3JvdXAocykuIFRoZSBwb3N0LWhvYyBgZHVubi50ZXN0YCBpcyByZWNvbW1lbmRlZCB3aGljaCBhbHNvIHBlcmZvcm1zIG11bHRpcGxlIHRlc3RpbmcgY29ycmVjdGlvbi4KCmBgYHtyfQpkaWV0ICU+JSAKICBkdW5uX3Rlc3Qod2VpZ2h0Lmxvc3MgfiBkaWV0LnR5cGUsIHAuYWRqdXN0Lm1ldGhvZCA9ICJib25mZXJyb25pIikKYGBgCgoKQXQgdGhpcyBwb2ludCB3ZSBjb3VsZCBiZSBhYm91dCB0byByZWNvbW1lbmQgZGlldCBDIHRvIHRob3NlIHRoYXQgd2lzaCB0byBsb3NlIHdlaWdodC4gSG93ZXZlciwgYXJlIHRoZXJlIGFueSBvdGhlciBmYWN0b3JzIGluIHRoZSBkYXRhIHRoYXQgd2Ugc2hvdWxkIGJlIGNvbnNpZGVyaW5nPyBXaXRoIGBnZ3Bsb3QyYCB3ZSBjYW4gcXVpdGUgZWFzaWx5IHZpc3VhbGlzZSB0aGUgZWZmZWN0cyBvZiBtdWx0aXBsZSBmYWN0b3JzIG9uIHRoZSBkYXRhLiBMZXRzIGFkZCBib3RoIGdlbmRlciBhbmQgZGlldCB0eXBlIGludG8gdGhlIHBsb3QuIEl0IG5vdyBhcHBlYXJzIHRoYXQgZGlldCBDIGlzIGhhdmluZyBhbiBlZmZlY3Qgb24gbWFsZXMgYnV0IG5vdCBmZW1hbGVzLgoKYGBge3J9CmdncGxvdChkaWV0LCBhZXMoeCA9IGRpZXQudHlwZSwgeSA9IHdlaWdodC5sb3NzLGZpbGw9Z2VuZGVyKSkgKyBnZW9tX2JveHBsb3QoKQpgYGAKCiMjIFR3by13YXkgQU5PVkEKClRoZSAqZm9ybXVsYSogbm90YXRpb24gYWxsb3dzIHVzIHRvIHNwZWNpZnkgYW4gKmludGVyYWN0aW9uKiBiZXR3ZWVuIGdlbmRlciBhbmQgZGlldCB0eXBlLiBJbiBvdGhlciB3b3Jkcywgd2UgYXJlIGxvb2tpbmcgdG8gc2VlIGlmIHRoZSBlZmZlY3Qgb2YgZGlldCB0eXBlIGlzIGRpZmZlcmVudCBmb3IgbWFsZXMgYW5kIGZlbWFsZXMuIEluIFIsIHRoZSBmb3JtdWxhIGZvciBhbiBpbnRlcmFjdGlvbiBpcyBzcGVjaWZpZWQgdXNpbmcgYSBgKmAgYmV0d2VlbiB0aGUgdmFyaWFibGVzIHRoYXQgd2UgYXJlIGludGVyZXN0ZWQgaW4gYXNzZXNzaW5nIHRoZSBpbnRlcmFjdGlvbiBmb3IuCgpgYGB7cn0KZGlldCAlPiUgCiAgYW5vdmFfdGVzdCh3ZWlnaHQubG9zcyB+IGRpZXQudHlwZSpnZW5kZXIpCmBgYAoKClRoaXMgdGVsbHMgdXMgdGhhdCBhbiBlZmZlY3QgZXhpc3RzIGJldHdlZW4gZGlldCB0eXBlIGFuZCBnZW5kZXIsIGJ1dCBsaWtlIGJlZm9yZSB3ZSBoYXZlIHRvIHJ1biBhIHBvc3QtaG9jIHRlc3QgdG8gZGlzY292ZXIgbW9yZQoKYGBge3J9CmRpZXQgJT4lIAogIHR1a2V5X2hzZCh3ZWlnaHQubG9zcyB+IGRpZXQudHlwZSpnZW5kZXIpCmBgYAoKCgoKCgojIyBSZXBlYXRlZCBNZWFzdXJlcwoKCioqQ0hFQ0sgVEhJUyBDT0RFIERPRVMgV0hBVCBXRSBXQU5UKioKCldlIHdpbGwgcmVhZCBhIG1vZGlmaWVkIHZlcnNpb24gb2YgdGhlIGRpZXQgZGF0YXNldCBpbiBvcmRlciB0byB0ZXN0IGEgInJlcGVhdGVkIG1lYXN1cmVzIiBhbmFseXNpcy4gSGVyZSB3ZSBoYXZlIGFkZGVkIGEgbWlkcG9pbnQgd2VpZ2h0IG1lYXN1cmVtZW50LgoKYGBge3J9CmRpZXQyIDwtIHJlYWRfY3N2KCJkYXRhL2RpZXQyLmNzdiIpClZpZXcoZGlldDIpCmBgYAoKSG93ZXZlciwgdGhlIHRocmVlIG1lYXN1cmVzIHRoYXQgd2Ugd2FudCB0byBjb21wYXJlIGFyZSBnaXZlbiBhcyBjb2x1bW5zIGluIHRoZSBkYXRhIGZyYW1lLiBXZSBjYW5ub3QgZXhwcmVzcyB0aGlzIHVzaW5nIHRoZSBSIGB+YCBub3RhdGlvbi4gSW4gb3RoZXIgd29yZHMgdGhlIGRhdGEgaXMgaW4gKndpZGUqIGZvcm1hdCBhbmQgbm90ICpsb25nKi4gV2UgY2FuIGNoYW5nZSB0aGlzIHVzaW5nIHRoZSBgdGlkeXJgIHBhY2thZ2UuIFRoaXMgY3JlYXRlcyBhIHZhcmlhYmxlIGZvciB0aGUgdHlwZSBvZiBtZWFzdXJlbWVudCAoYGluaXRpYWxgLCBgbWlkYCBvciBgZmluYWwpYCBhbmQgYSB2YXJpYWJsZSBjb250YWluaW5nIHRoZSBjb3JyZXNwb25kaW5nIHZhbHVlLiBUaGUgZGVmYXVsdCBjb2x1bW4gbmFtZXMgYXJlIGBuYW1lYCBhbmQgYHZhbHVlYCByZXNwZWN0aXZlbHkuCgpgYGB7cn0KIyMjIFdpdGggIjo6IiB5b3UgY2FuIHVzZSBhIGZ1bmN0aW9uIGZyb20gYSBwYWNrYWdlIHlvdSBoYXZlbid0IGxvYWRlZCB5ZXQKCmRpZXQyIDwtIGRpZXQyICU+JSBzZWxlY3QoaWQsZ2VuZGVyLCBjb250YWlucygid2VpZ2h0IikpICU+JSAKICB0aWR5cjo6cGl2b3RfbG9uZ2VyKDM6NSwgdmFsdWVzX3RvID0gInZhbHVlIiwgbmFtZXNfdG8gPSAidGltZV9wb2ludCIpICU+JSAKICBtdXRhdGUoaWQgPSBhcy5mYWN0b3IoaWQpKQpgYGAKCmBgYHtyfQphbm92YV90ZXN0KGRpZXQyLCBkdiA9dmFsdWUsIAogICAgICAgICAgIHdpZCA9IGlkLCAKICAgICAgICAgICB3aXRoaW4gPSB0aW1lX3BvaW50KQpgYGAKCmBgYHtyfQoKYGBgCgoKCjxkaXYgY2xhc3M9ImFsZXJ0IGFsZXJ0LXdhcm5pbmciPgoqKkV4ZXJjaXNlKioKClRoZSBleGNlbCBmaWxlIOKAmFJDQzLigJkgY29udGFpbnMgZGF0YSBhYm91dCB0aGUgZXhwcmVzc2lvbiBsZXZlbHMgb2Ygc29tZSBnZW5lcyBpbiBwYXRpZW50cyB3aXRoIHJlbmFsIGNlbGwgY2FyY2lub21hLiBJbiB5b3VyIHN0dWR5LCB5b3UgcHV0IHRoZSBmb2xsb3dpbmcgaHlwb3RoZXNlcy4KUGxlYXNlIHRlc3QgdGhvc2UgYWx0ZXJuYXRpdmUgaHlwb3RoZXNlcyBhbmQgc3RhdGUgd2hldGhlciB5b3Ugd2lsbCBhY2NlcHQgb3IgcmVqZWN0IGVhY2ggb25lLgoKMS4JRmVtYWxlcyBoYXZlIGEgaGlnaGVyIGxldmVsIG9mIEUyRjMgdGhhbiBtYWxlcyAKMi4JQU5YQSBleHByZXNzaW9uIGxldmVscyB2YXJ5IGJldHdlZW4gdW5pbGF0ZXJhbCBhbmQgYmlsYXRlcmFsIHR1bW9ycwozLglJbmRpdmlkdWFscyB3aXRoIFJDQyBncmFkZSBJSSBoYXZlIGRpZmZlcmVudCBsZXZlbHMgb2YgRTJGMyB0aGFuIHRob3NlIHdpdGggZ3JhZGUgSUlJIG9yIElWIAo0LglUaGUgbWVhbiB2YWx1ZSBvZiBtaVI0OTkgZGVjcmVhc2VzIHNpZ25pZmljYW50bHkgYWZ0ZXIgdHJlYXRtZW50CjUuCURGRkEgaXMgaGlnaGVyIGluIHBhdGllbnRzIHdpdGggZ3JhZGUgSVYgdHVtb3JzCjwvZGl2PgoKIyBTb2x1dGlvbnMKCioqVG8gYmUgcmV2ZWFsZWQgZHVyaW5nIHRoZSB3b3Jrc2hvcCEqKgoKKipOb3RlIHRoYXQgdGhlIGNvZGUgaXMgaW5jbHVkZWQgZnJvbSB0aGUgb2xkZXIgdmVyc2lvbiBvZiB0aGUgY291cnNlICh1c2luZyBiYXNlIFIgYW5kIGdncGxvdDIpIGFzIHdlbGwgYXMgcnN0YXRpeCBhbmQgZ2dwdWJyIHRvIGNoZWNrIHdlIGdldCB0aGUgc2FtZSBhbnN3ZXJzKioKCiMjIEV4ZXJjaXNlIDEKCmBgYHtyfQojIyNPTEQKCiMjMS1SZWFkIHRoZSBleGNlbCBmaWxlIGNhbGxlZCAnRVggQmlvc3RhdCBQMScgaW50byBSIApCaW9zdGF0MSA8LSByZWFkeGw6OnJlYWRfeGxzeCgiZGF0YS9FWCBCaW9zdGF0IFAxLnhsc3giKQoKIyMyLU1ha2UgYSBjcm9zcyB0YWJsZSBzaG93aW5nIHRoZSBnZW5kZXIgaW4gdGhlIHJvd3MgYW5kIHR1bW9yIGdyYWRlIGluIHRoZSBjb2x1bW5zCkdlbmRlckdyYWRlIDwtIHRhYmxlKEJpb3N0YXQxJEdlbmRlciwgQmlvc3RhdDEkYFR1bW9yIGdyYWRlYCkKR2VuZGVyR3JhZGUKCiMjMy1EZWZpbmUgdGhlIHBlcmNlbnRhZ2Ugb2YgR3JhZGUgSUlJIHR1bW9ycyB3aXRoaW4gZmVtYWxlcyAKcHJvcC50YWJsZShHZW5kZXJHcmFkZSwxKSoxMDAgIyg0Mi4zJSkKCiMjNC1BZGQgYSBjb2x1bW4gaW4gdGhlIHRhYmxlIHNob3dpbmcgdGhlIHN1bSBvZiB0aGUgMyBncmFkZXMgaW4gbWFsZXMgYW5kIGluIGZlbWFsZXMgYW5kIHN0YXRlIHRoZSB0b3RhbCBudW1iZXIgb2YgbWFsZXMgYW5kIGZlbWFsZXMgaW4gdGhlIHNhbXBsZSAKYWRkbWFyZ2lucyhHZW5kZXJHcmFkZSwyKSAjKDcxIEZlbWFsZXMsIDU5IG1hbGVzKQoKIyM1LVVzZSB0aGUgYXBwcm9wcmlhdGUgdGVzdCB0byBjaGVjayBpZiB0aGUgdHVtb3IgZ3JhZGUgZGVwZW5kcyBvbiB0aGUgZ2VuZGVyIApjaGlzcS50ZXN0KEdlbmRlckdyYWRlKSAjZ3JhZGUgZGVwZW5kcyBvbiBnZW5kZXIgKHNpZ25pZmljYW50KQpgYGAKCmBgYHtyfQojIyBORVcKCkJpb3N0YXQxIDwtIHJlYWR4bDo6cmVhZF94bHN4KCJkYXRhL0VYIEJpb3N0YXQgUDEueGxzeCIpCiMjMi1NYWtlIGEgY3Jvc3MgdGFibGUgc2hvd2luZyB0aGUgZ2VuZGVyIGluIHRoZSByb3dzIGFuZCB0dW1vciBncmFkZSBpbiB0aGUgY29sdW1ucwoKY291bnQoQmlvc3RhdDEsIEdlbmRlciwgYFR1bW9yIGdyYWRlYCkKCgojIzMtRGVmaW5lIHRoZSBwZXJjZW50YWdlIG9mIEdyYWRlIElJSSB0dW1vcnMgd2l0aGluIGZlbWFsZXMgCmZyZXFfdGFibGUoQmlvc3RhdDEsIEdlbmRlciwgYFR1bW9yIGdyYWRlYCkKCiMjNC1Vc2UgdGhlIGFwcHJvcHJpYXRlIHRlc3QgdG8gY2hlY2sgaWYgdGhlIHR1bW9yIGdyYWRlIGRlcGVuZHMgb24gdGhlIGdlbmRlciAKCmNvdW50KEJpb3N0YXQxLCBHZW5kZXIsIGBUdW1vciBncmFkZWApICU+JSAKICAgIHRpZHlyOjpwaXZvdF93aWRlcihldmVyeXRoaW5nKCksIG5hbWVzX2Zyb20gPSAiR2VuZGVyIix2YWx1ZXNfZnJvbSA9ICJuIikgJT4lIAogIHNlbGVjdCgtYFR1bW9yIGdyYWRlYCkgJT4lIAogIGNoaXNxX3Rlc3QoKQoKYGBgCgojIyBFeGVyY2lzZSAyCgoKYGBge3J9CgojIyBPTEQKCiMjMS1SZWFkIHRoZSBleGNlbCBmaWxlIGNhbGxlZCAnRVggQmlvc3RhdCBQMicgaW50byBSIApCaW9zdGF0MiA8LSByZWFkeGw6OnJlYWRfeGxzeCgiZGF0YS9FWCBCaW9zdGF0IFAyLnhsc3giKQoKaGlzdChCaW9zdGF0MiRBZ2UsZnJlcSA9IEZBTFNFKQpsaW5lcyhkZW5zaXR5KEJpb3N0YXQyJEFnZSkpCgojIzItSWRlbnRpZnkgaWYgdGhlIGFnZSBhbmQgaG9zcGl0YWxpemF0aW9uIGRheXMgYXJlIG5vcm1hbGx5IGRpc3RyaWJ1dGVkCmxpYnJhcnkoZ2dwbG90MikKZ2dwbG90KEJpb3N0YXQyLCBhZXMoeD1BZ2UpKSArIGdlb21faGlzdG9ncmFtKGFlcyh5PS4uZGVuc2l0eS4uLCksYmlud2lkdGggPSAyKSArIGdlb21fZGVuc2l0eSgpCmdncGxvdChCaW9zdGF0MiwgYWVzKHg9IiIseT1BZ2UpKSAgKyBnZW9tX3Zpb2xpbigpICsgZ2VvbV9ib3hwbG90KCkKZ2dwbG90KEJpb3N0YXQyLGFlcyhzYW1wbGU9QWdlKSkgKyBnZW9tX3FxKCkgKyBnZW9tX3FxX2xpbmUoKQpzaGFwaXJvLnRlc3QoQmlvc3RhdDIkQWdlKQojIEFnZSBub3JtYWxseSBkaXN0cmlidXRlZAoKCmdncGxvdChCaW9zdGF0MiwgYWVzKHg9YGhvc3BpdGFsaXphdGlvbiBkYXlzYCkpICsgZ2VvbV9oaXN0b2dyYW0oYWVzKHk9Li5kZW5zaXR5Li4pKSArIGdlb21fZGVuc2l0eSgpCmdncGxvdChCaW9zdGF0MiwgYWVzKHg9IiIseT1gaG9zcGl0YWxpemF0aW9uIGRheXNgKSkgICsgZ2VvbV92aW9saW4oKSArIGdlb21fYm94cGxvdCgpCmdncGxvdChCaW9zdGF0MixhZXMoc2FtcGxlPWBob3NwaXRhbGl6YXRpb24gZGF5c2ApKSArIGdlb21fcXEoKSArIGdlb21fcXFfbGluZSgpCnNoYXBpcm8udGVzdChCaW9zdGF0MiRgaG9zcGl0YWxpemF0aW9uIGRheXNgKQoKIyBob3NwaXRhbGl6YXRpb24gZGF5cyBOT1Qgbm9ybWFsbHkgZGlzdHJpYnV0ZWQKCiMjMy1Vc2UgdGhlIGFwcHJvcHJpYXRlIGRlc2NyaXB0aXZlIHN0YXRpc3RpY3MgW21lYW4rLVNEIG9yIG1lZGlhbiAoSVEgcmFuZ2UpXSBmb3IgZWFjaCB2YXJpYWJsZQpsaWJyYXJ5KHRpZHl2ZXJzZSkKc3VtbWFyaXNlKEJpb3N0YXQyLCBNZWFuQWdlID0gbWVhbihBZ2UpLCBzZEFnZSA9IHNkKEFnZSksIAogICAgICAgICAgbWVkaWFuSG9zcERheXM9bWVkaWFuKGBob3NwaXRhbGl6YXRpb24gZGF5c2ApLCAKICAgICAgICAgIFExPXF1YW50aWxlKGBob3NwaXRhbGl6YXRpb24gZGF5c2AsIDAuMjUpLCAKICAgICAgICAgIFEzPXF1YW50aWxlKGBob3NwaXRhbGl6YXRpb24gZGF5c2AsIDAuNzUpKQojQWdlOiA5LjU0ICsvLSAyLjg5IFttZWFuICstIFNEXSAgJiBIb3NwaXRhbGl6YXRpb24gZGF5czogMTEuNSAoOS4wMCAtIDE0LjI1KSBbbWVkaWFuKElRIHJhbmdlKV0KYGBgCgpgYGB7cn0KIyMjTkVXCgojIzEtUmVhZCB0aGUgZXhjZWwgZmlsZSBjYWxsZWQgJ0VYIEJpb3N0YXQgUDInIGludG8gUiAKQmlvc3RhdDIgPC0gcmVhZHhsOjpyZWFkX3hsc3goImRhdGEvRVggQmlvc3RhdCBQMi54bHN4IikKCmdnaGlzdG9ncmFtKEJpb3N0YXQyLCB4ID0gIkFnZSIpCgpnZ2hpc3RvZ3JhbShCaW9zdGF0MiwgeCA9ICJBZ2UiLCBhZGRfZGVuc2l0eSA9IFRSVUUsIHkgPSAiLi5kZW5zaXR5Li4iKSArIHN0YXRfb3ZlcmxheV9ub3JtYWxfZGVuc2l0eShjb2w9ImJsdWUiKQoKCmdncXFwbG90KEJpb3N0YXQyLCB4ID0gIkFnZSIpCkJpb3N0YXQyICU+JSAKICBzaGFwaXJvX3Rlc3QodmFycyA9ICJBZ2UiKQojIEFnZSBpcyBub3JtYWxseSBkaXN0cmlidXRlZAoKZ2doaXN0b2dyYW0oQmlvc3RhdDIsIHggPSAiaG9zcGl0YWxpemF0aW9uIGRheXMiKQoKZ2doaXN0b2dyYW0oQmlvc3RhdDIsIHggPSAiaG9zcGl0YWxpemF0aW9uIGRheXMiLCBhZGRfZGVuc2l0eSA9IFRSVUUsIHkgPSAiLi5kZW5zaXR5Li4iKSArIHN0YXRfb3ZlcmxheV9ub3JtYWxfZGVuc2l0eShjb2w9ImJsdWUiKQoKCmdncXFwbG90KEJpb3N0YXQyLCB4ID0gImhvc3BpdGFsaXphdGlvbiBkYXlzIikKQmlvc3RhdDIgJT4lIAogIHNoYXBpcm9fdGVzdCh2YXJzID0gImhvc3BpdGFsaXphdGlvbiBkYXlzIikKCiMgaG9zcGl0YWxpemF0aW9uIGRheXMgTk9UIG5vcm1hbGx5IGRpc3RyaWJ1dGVkCgpnZXRfc3VtbWFyeV9zdGF0cyhCaW9zdGF0MikKCmBgYAoK